[SystemSafety] Autonomous Vehicles and "Hacking" Threats

[Peter Bernard Ladkin]

On 2014-11-21 17:38 , Stefan Winter wrote:
> ......
> I had hoped for some better estimate of defect densities ...... The best approximation I
> had come up with so far is the product of "lines of code in a modern car" (100 million for a premium
> car in 2009) and "defect count per line of code in really critical software" (10^-4).....
> If anyone has a better idea or wants to share more accurate numbers, please let me know.

Is your interest strictly automotive, or in gathering information from various sectors? Assuming
it's the latter, here is some more info.

The main problem in gathering good statistics, in any sector, is that it mostly has to be hearsay.
Companies are loathe to suggest or admit, or even in some cases contemplate, that their critical SW
is anything but perfect. For reasons of liability and consumer trust. (I expect the first slide in a
business talk about developing client trust to say just one thing: "Earn it!". I am mostly
disappointed.)

Banks, for example, are not going to let you know how anything about how buggy or vulnerable their
ATM code may be or may have been (see Ross Anderson's writing on the John Munden case in the Risks
Form and on his WWW site). I have not yet indulged in on-line banking with my local bank because
continuing requests for information about how secure their systems are has been met with promises to
have someone contact me, and then silence, for a decade. Even though they had a glitch a few years
ago about suspending access because of a code breach that made it onto the front page of the local
newspaper, they are unwilling to share information: "Oh, that! That was precaution - none of our
customers lost any money." They used to send out by post pseudo-one-time-pads, called TANs. After
the "glitch" they abandoned that system. So, I said, that surely shows us there was a structural
vulnerability that they couldn't fix, and it's that kind of stuff that concerns me. "Oh, no, we were
testing our new system and just thought it was a good time to transition." The new system being one
that sends your next pseudo-one-time access code to your mobile phone.

My thoughts on that are as follows. Whatever "glitch" it was, it was not somebody at the post office
steaming open envelopes containing the next batch of your numbers. Because they couldn't use them
unless they know what your account details are, and what your primary access information is, namely
your logon password (they know who you are, of course, from the name and address). So it's something
with someone who knows all that information, but who is just lacking the pseudo-one-time-access
code. If you can figure out how those are generated, and it's not that random, then you can execute
playback attacks quite easily. Ross Anderson and co (again) have a paper on how secure many of the
ATM systems with pseudo-one-time-pads around Cambridge actually are (answer: not by any means as
much as you might like).

Pre-Internet, someone in an authoritative position to know told me in the mid-1990's that the cost
of fraudulent activity in interbank electronic transfer systems was, as he said, higher than my
worst guess, whatever that guess might be.

John McDermid and Tim Kelly have a paper from 2006, Software in Safety Critical Systems: Achievement
and Prediction (Nuclear Safety 02(03), 2006), available from York's WWW site, in which they have a
section Industry Data with subsections Fault Density and Failure Rates, which discuss just that.
Here's a version; I don't know if it is identical with the published one
http://www-users.cs.york.ac.uk/tpk/inuce2004.pdf wait a mo, here is the published version from
John's page:
ftp://ftp.cs.york.ac.uk/pub/hise/Software_in_Safety_Critical_Systems:Achievement_and_Prediction.pdf

PBL

Prof. Peter Bernard Ladkin, Faculty of Technology, University of Bielefeld, 33594 Bielefeld, Germany
Tel+msg +49 (0)521 880 7319  www.rvs.uni-bielefeld.de