[SystemSafety] Automobile Safety-Critical Kit (Bookout v. Toyota Motor transcript)

[Martyn Thomas]

Heath

You may be right that the code had nothing to do with the crash but if
the outcome is much better software engineering then let's celebrate
that. Good engineering is free - there may be some training costs but
the ROI is high and you will save money every year following, for ever.
If you need to convince someone, get them to look at the Tokeneer
project for NSA (all details on line) and compare their productivity and
defect densities.

Quality is free - really! - I have seen it (and I bet my own company on
it over many years).

Martyn

On 31/10/2013 20:38, Heath Raftery wrote:
> I read the transcript as best I could yesterday. To read it word for
> word would likely take a couple of days, so I skimmed. Interestingly,
> the excerpts in the EE Times article I did read verbatim - I think
> that section was not only of greatest interest to engineers, but
> probably ended up having the greatest impact on the court.
>
> Mr Barr seems, based on the transcript, to be an excellent code
> reviewer. He's also an excellent communicator. He put together a very
> convincing case that the code was unsatisfactory. Not only was the
> quality of the code poor, their review and tracking procedures were
> sorely lacking. This is beneath what we would want to believe a car
> manufacturer adheres to.
>
> On the other hand, I remain unconvinced the software had anything to
> do with the crash. The driver was 76 years old at the time. This crash
> was subject to an NTSB investigation, and investigators found no
> evidence that it was a software fault or a hardware fault. The crash
> recorder says the driver pushed the accelerator and was not pushing
> the brakes, and then the car was hit. Even Mr Barr's demonstration of
> a potential fault scenario (inject a bit flip, task dies, cruise
> control is resumed while the brake is lightly applied - acceleration
> continues past setpoint), while clearly a bug that should be fixed, is
> very hard to imagine as relevant to the crash.
>
> I believe Mr Barr rightly convinced the court that the software
> development project lacked quality, and then, by virtue of some
> terrible cross-examination, he leads the jury to assume that the
> software bugs could have caused the accident.
>
> I suspect, with no proof, that the code reached that state iteratively
> - I could well imagine it started with the best intentions and best
> practices. Then, as budgets dwindled, management got jumpy, and
> feature request/bug fixes got thrown in at the last minute, the code
> got messy and the review process fell away. The team probably consoled
> themselves by enormous system tests - in 1000's of km of testing, not
> one issue, so the crappy code works. It looks like rubbish, but it works.
>
> Then, an old lady hits the accelerator instead of the brake, the code
> gets presented in court, a jury of laypeople looking for someone to
> blame are shocked at what actually runs the cars we drive and boom -
> Toyota is at fault in an accident causing death.
>
> In the end I think people will have to start to expect software to
> cost more - we need these laborious (and rather dull, frankly) safety
> practices for two reasons: the first is that they can help build in
> quality (cue endless argument about whether they are the best way or
> even an effective way) and second because as engineers we're
> responsible for people's lives and sooner or later we need *evidence*
> that we've taken that into consideration.
>
> It's an incredibly timely episode for our workplace, where we develop
> hardware and firmware for safety critical applications in another
> field entirely (mining) - there's this uneducated push to "go SIL" but
> no one appreciates the learning curve and the cost, and when push
> comes to shove, we just resort back to getting the job done as quickly
> as possible. That's starting to change.
>
> Regards,
> Heath
>
> On 1/11/2013 5:30 AM, Chris Hills wrote:
>> The complete transcript.
>>
>> http://veriloud.com/Barr_REDACTED.pdf
>>
>> *From:*systemsafety-bounces at lists.techfak.uni-bielefeld.de
>> [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] *On Behalf
>> Of *Chuck_Petras at selinc.com
>> *Sent:* 31 October 2013 17:57
>> *To:* systemsafety at techfak.uni-bielefeld.de
>> *Subject:* Re: [SystemSafety] Automobile Safety-Critical Kit (Bookout v.
>> Toyota Motor transcript)
>>
>> Some excerpts from the court transcript...
>>
>> Toyota Case: Inside Camry’s Electronic Control Module,
>> <http://www.eetimes.com/document.asp?doc_id=1319952>
>>
>>
>> Chuck Petras, PE**
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>