[SystemSafety] Automobile Safety-Critical Kit (Bookout v. Toyota Motor transcript)

[Peter Bernard Ladkin]

Following this up a little more, there is also an interesting earlier article in EETimes at
http://www.eetimes.com/document.asp?doc_id=1319903

It might be useful to have someone summarise what the Barr Group report identified in its code
inspection. I don't know whether it is just me, but I haven't managed to get a clear idea yet.

I just looked at the transcript pp70-85. I'll refer to "Barr" to mean any of the experts working
with the Barr Group. Barr himself gave the testimony in this transcript.

* Apparently there is no HW error detection and correction ("EDAC" they say here) on the control
board for this car (a 2005 Camry). I particularly like the reference to a "parody bit" on p72.

* Apparently Barr identified potential for stack overflow and ensuing memory corruption in the
operation of the SW

* There is a high-priority task, which Barr refers to as "Task X" or the "kitchen-sink task" which
performs throttle control (that is, sets throttle angle); "executes" cruise control code (which
turns cruise control on and off, and maintains speed when it is working); is "responsible for"
various "failsafe" procedures on the "main CPU"; is responsible for raising the Diagnostic Trouble
Codes (DTCs). On later models, for example the 2010 Camry, there is a brake override function, which
"cuts the throttle" (presumably commands throttle angle to closed) when the brake pedal is
depressed. There is no such function on the car in question, the 2005 Camry. But in the 2010 Camry
the brake override is in Task X, according to Barr.

* There is a possibility that Task X might "die". I don't know whether that means terminated, or
unable to progress.

* One of the ways in which it might do this is through a "bit flip". I think this is their term for
an ostensibly-uncommanded change in value of a bit. Presumably they are thinking about the bit which
designates active or not in the process table. I haven't found a place in which they talk about OS
architectures and process tables, though; there is no occurrence of "process table" in the
transcript. It appears that Barr performed fault-injection until they got Task X to cease
progressing, but this is not 100% clear from the transcript.

So it looks as if Task X performs not only command functions for the throttle and calls cruise
control code, but it is also responsible for what IEC 61508 calls safety functions. Apparently it is
also responsible for the "brake override" safety function in later models of the car.

That Task X can be terminated, or unable to progress, is obviously problematic. That there is no HW
EDAC is also very surprising in something with critical function brought to the market in 2005. That
Task X includes a safety function in 2010 should preclude that it can be terminated or unable to
progress in its 2010 version.

On 10/31/13 9:38 PM, Heath Raftery wrote:
> On the other hand, I remain unconvinced the software had anything to do with the crash. The driver
> was 76 years old at the time. This crash was subject to an NTSB investigation, and investigators
> found no evidence that it was a software fault or a hardware fault. The crash recorder says the
> driver pushed the accelerator and was not pushing the brakes, and then the car was hit. Even Mr
> Barr's demonstration of a potential fault scenario (inject a bit flip, task dies, cruise control is
> resumed while the brake is lightly applied - acceleration continues past setpoint), while clearly a
> bug that should be fixed, is very hard to imagine as relevant to the crash.

The comment on the Beasley Allen WWW site makes much of the skid marks.

Can you cite the document which says that the crash recorder records accelerator-pedal depression
and no brak-pedal depression?

> I believe Mr Barr rightly convinced the court that the software development project lacked quality,
> and then, by virtue of some terrible cross-examination, he leads the jury to assume that the
> software bugs could have caused the accident.

What could the cross-examiner have done? Barr has apparently established defects in the code, and
the only counter would be that *those* defects were not active during the accident in question. As
Barr says, you would be trying to establish a negative, which can't be done very well on the limited
product testing in the automotive domain, as we have discussed regularly on the York list.
Operational use vastly exceeds the time available for targeted manufacturer testing, or any other kind.

Barr used fault-injection techniques, which is an obvious choice if the kit isn't using EDAC, and he
found faults which allow UA. Ipso facto, they exist. How on earth are you going to establish that
they didn't manifest in the specific accident under review? I imagine the manufacturer was well
aware of what was going on, and had no suitable way of responding.

Is there indeed any suitable response?

I think that shows the significance of this case. BTW, it is established case law - it won't go to
appeal. The manufacturer has settled with the plaintiffs.

It turns out one of the expert witnesses is a colleague at CMU. He can't talk about the case.

It is also notable that the only part of the transcript (yet) available is the examination and
cross-examination of Michael Barr. I wonder how that got out there? It does show the power of
selective release. None of the Toyota expert evidence is available; neither is any of the evidence
of expert witnesses other than that of Mr. Barr.

> In the end I think people will have to start to expect software to cost more - we need these
> laborious (and rather dull, frankly) safety practices for two reasons:

I agree with Martyn that effective software-quality enhancement practices need not cost more. I
think what might cost more, initially, is the kit to support it. If you want EDAC in your HW, this
costs more per delivered vehicle. If you want to use software development practices that exclude
Dependability1 errors, amongst other things you'll likely need a compiler for a strongly-typed
language which targets the chipset you're working with. On current practice, such compilers may or
may not exist.

Indeed, people reading this transcript may well decide that putting safety functions into an
"kitchen-sink" monitor task whose progress can be halted during fault-injection testing is an
industry habit that must be discontinued forthwith, given that it has been so deprecated in a court.

PBL

Prof. Peter Bernard Ladkin, Faculty of Technology, University of Bielefeld, 33594 Bielefeld, Germany
Tel+msg +49 (0)521 880 7319  www.rvs.uni-bielefeld.de