[SystemSafety] Autonomously Driven Car Kills Pedestrian

Tue Mar 20 15:10:16 CET 2018

To demonstrate compliance with ISO 26262, an organisation is required to "create, foster, and sustain a safety culture that supports and encourages the effective achievement of functional safety" (ISO 26262-2: 2011, Clause 5.4.2.1).

I work with many organisations (in different sectors) where there is – in my view - little or no evidence of such a culture.  

In my view, a key challenge we face – as a society – is to work out how we can best encourage organisations that develop safety-related systems to embrace such a culture.  This has – of course – been the focus of a related discussion on this list in recent weeks.

We’ve had such discussions before.  If I recall correctly, PBL proposed previously that we should use what are in the UK called ‘Chartered Engineers’ to develop such systems, one reason being that the chartered status could be removed if it was demonstrated that a safety-related system failed to operate correctly because ‘good engineering practice’ had not been applied during the development.  This sounds like a promising starting point to me.

In my view, everything then follows from the safety culture – team recruitment, choice of standards, choice of hardware, software, etc.

Others may - of course - see the world differently.

Michael J. Pont

SafeTTy Systems.

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Steve Tockey
Sent: 20 March 2018 13:19
To: Nick Tudor <njt at tudorassoc.com>
Cc: systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] Autonomously Driven Car Kills Pedestrian

If it were up to me—and it is clearly not—I would require all code for self-driving functions to show compliance with at least Level C of DO-178C / ED-12C before that car can be put on a public road. 

My 2 cents,

— steve 

发自我的 iPad

On Mar 20, 2018, at 6:15 AM, Nick Tudor <njt at tudorassoc.com <mailto:njt at tudorassoc.com> > wrote:

Hmmm.  ISO26262 is not mandated (SFIK anywhere) and does not cover autonomous cars.  It is also left to the developer to make an argument as to why their systems are safe based upon some subjective activities 'Recommended' etc , by ISO26262.  There is no requirement for independent scrutiny, so a developer can make up any old story and say their product meets ISO26262 (perhaps this is what is going on in the AV world...?) Hence, in my view, it is not particularly 'safe'....and wouldn't add much to the AV world.

Nick Tudor 

Tudor Associates Ltd

Mobile: +44(0)7412 074654

www.tudorassoc.com <http://www.tudorassoc.com> 

77 Barnards Green Road

Malvern

Worcestershire

WR14 3LR
Company No. 07642673

VAT No:116495996

www.aeronautique-associates.com <http://www.aeronautique-associates.com>  

On 20 March 2018 at 13:04, Andrew Banks <andrew at andrewbanks.com <mailto:andrew at andrewbanks.com> > wrote:

Agreeing with David (shock horror!) it should also be noted that, in the UK, the official policy of the Department for Transport (DfT) is not to regulate as “this will stifle innovation”, and have explicitly ruled out mandating ISO 26262 (for non-autonomous as well as autonomous) unless and until UNECE do so.

IMHO this policy is very worrying…

A

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de <mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de> ] On Behalf Of David Ward
Sent: 20 March 2018 12:36

To: systemsafety at lists.techfak.uni-bielefeld.de <mailto:systemsafety at lists.techfak.uni-bielefeld.de> 
Subject: Re: [SystemSafety] Autonomously Driven Car Kills Pedestrian

There is an SAE document SAE J3018 “Guidelines for Safe On-Road Testing of SAE Level 3, 4, and 5 Prototype Automated Driving Systems (ADS)” which includes reference to a “Safety development process”.  While this doesn’t explicitly refer to ISO 26262 some of the concepts are very similar.

David Ward

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Martyn Thomas
Sent: 20 March 2018 12:31
Cc: systemsafety at lists.techfak.uni-bielefeld.de <mailto:systemsafety at lists.techfak.uni-bielefeld.de> 
Subject: Re: [SystemSafety] Autonomously Driven Car Kills Pedestrian

Has anyone actually defined what evidence should be required before a level 5 AV should be licenced?

Regards

Martyn

On 20 Mar 2018, at 12:25, Tom Ferrell <tom at faaconsulting.com <mailto:tom at faaconsulting.com> > wrote:

I agree that legislative bodies should not be waiving standards in general.  However, as I understand it, there are numerous stipulations in place on the self-driving trials underway in AZ including extensive data collection that were put in place by government.  Given the magnitude of what this technology is trying to accomplish, demonstration on closed tracks, through simulation, or controlled experiments is simply not going to be enough.  How long do you propose waiting before these vehicles can be on public streets?

As for the claims relating to lives saved:  This is exactly the same argument that has been made for a host of other ‘safety-enhancement’ systems and devices.  The issue here is that it removes the human from the immediate control loop, a step further than systems like TCAS that ultimately had to be mandated by Congress given the pushback from the airframers and pilots.  The success of that system was quickly evident and is no longer disputed. 

I agree that the burden to prove this technology is safe is the designers/manufacturer’s to bear.  I am just saying that we need to stay focused on clearly articulating why we feel the trials or approaches being taken are technically in error or inadequate.  Otherwise, our voices will just get lumped in with the luddites pushing back against change.

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of C. Michael Holloway
Sent: Tuesday, March 20, 2018 8:02 AM
To: systemsafety at lists.techfak.uni-bielefeld.de <mailto:systemsafety at lists.techfak.uni-bielefeld.de> 
Subject: Re: [SystemSafety] Autonomously Driven Car Kills Pedestrian

... This revolution is coming whether we like it or not.  ...

Well, if it should not be coming yet, it is our duty to do everything we can to slow it down to a reasonable pace. That's the point that Prof. Cummings was making in the WaPO article. There are a bunch of ridiculous claims being made by the self-driving car zealots, which are unfortunately being accepted by the US Congress. The most egregious is the assertions about the number of lives that can saved. That's not a reason it is an excuse. If saving lives was the motivation, there are far simpler ways to accomplish it. 

What we should be asking is whether the systems being employed in these vehicles have been developed correctly in accordance with ISO26262 or similar standard. 

The US Congress exempted self-driving cars from having to meet *any* standards. That should not have happened, regardless of whether this particular accident turns out to have been the automation's fault.

-- 
All the best,
C. Michael Holloway (cMh)
Senior Research Computer Engineer
NASA Langley Research Center, Hampton VA USA
bit.ly/cmhpubs <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbit.ly%2Fcmhpubs&data=02%7C01%7Cdavid.ward%40horiba-mira.com%7C2472b6f7eed84da3967908d58e5e7b19%7Caa85aed398b34cdab14015ccbb32c3b5%7C1%7C1%7C636571458803641760&sdata=1YRrQ7QbHBR7XY%2FDwFahs0z5RGiPY9C5wKSzAp6K%2B9M%3D&reserved=0>  

Verba volant, scripta manent 
spoken words fly away, written words remain

(The words in this message are mine alone;
neither blame nor credit NASA for them.)

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE <mailto:systemsafety at TechFak.Uni-Bielefeld.DE> 

HORIBA MIRA Ltd

Watling Street, Nuneaton, Warwickshire, CV10 0TU, England

Registered in England and Wales No. 9626352

VAT Registration  GB 100 1464 84

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE <mailto:systemsafety at TechFak.Uni-Bielefeld.DE> 

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE <mailto:systemsafety at TechFak.Uni-Bielefeld.DE> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180320/626b7a99/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 425 bytes
Desc: not available
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180320/626b7a99/attachment-0001.jpg>