[SystemSafety] Uber Advanced Technologies Group publishes its "Safety Case Framework"

[Grazebrook, Alvery AN]

Am I missing something, or does the safety case completely fail to address the human-in-the-loop question? There are references to potential harm from reasonably foreseeable misuse, which is part of the story, but is there anything in the document to ensure that driver-in-the-loop involvement is within the capacity of a normal (trained or untrained as appropriate) human to act on? It may be that this is a moot point because they are only developing fully autonomous vehicles with no safety related human interaction outside misuse, but there’s nothing that says so.

Cheers,
            Alvery

** opinions expressed are my own, and not necessarily those of my employer.

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Bruce Hunter
Sent: 23 July 2019 03:32
To: systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] Uber Advanced Technologies Group publishes its "Safety Case Framework"

To be fair to Uber ATG, this is only meant to be the first layers and goals are developed only at a summary level. Although it misses the supporting strategy or context, it is good that they have gone public with this but it needs wider scrutiny and judgement against accepted standards.

Looking at the cybersecurity goals in G4.3 ("Potential harm from cyber intrusion is appropriately mitigated") does miss some critical elements but is a good start.
An interesting conflict point is in G4.3.4.2 "Over the air updates minimise duration of cyber vulnerabilities". Updates may be just as threatening to operational safety as the intrusions they are meant to protect against.

It is a shame this Safety Case was not completed and agreed before Uber AV trails were started. You wouldn't get away with it in other industries.

I hope Uber engages the wider professional community to truly validate the safety of their autonomous vehicle deployment. To quote Uber's next scary step "Melbourne, Dallas and Los Angeles are becoming the first cities to offer Uber Air flights, with the goal of beginning demonstrator flights in 2020 and commercial operations in 2023". Not sure if this safety-case approach is sufficient for the current road-based Uber AV let alone Uber Elevate project.

Bruce

On Tue, 23 Jul 2019 at 02:14, Martyn Thomas <martyn at thomas-associates.co.uk<mailto:martyn at thomas-associates.co.uk>> wrote:
It doesn't come close to the hard questions, such as "how safe is safe
enough?" and "what evidence would be sufficient to show with very high
confidence that this level of safety has been achieved?".

So far, they have published the goals. I'd like to see one of the
hardest goals (cybersecurity for example, or even just "securing the
supply chain") taken into enough detail that it exposes how this will be
done and explains how the assurance will be sufficient for a
safety-critical system.

Why does Uber think they have to reinvent all this? Why not start with
the processes that are used by Airbus, Siemens and Boeing, say?  And
update them with the special problems that autonomous cars face (such as
high bandwidth connectivity, machine learning, and cyclists).

If they don't build explicitly and rigorously on what other
safety-critical systems engineers have done, they will have to repeat
decades of work and failures.

Martyn


On 22/07/2019 14:58, Paul Sherwood wrote:
> Colleagues referred me to an article [1] describing Uber's "...
> blueprint for safe self-driving cars it wants the rest of the industry
> to follow".
>
> I think the article is an interesting read in itself, separate from
> the Safety Case website [2] and Uber's own article about the approach [3]
>
> [1]
> https://qz.com/1667964/ubers-launched-its-safety-case-for-self-driving-cars/
> [2] https://uberatg.com/safetycase
> [3]
> https://medium.com/@UberATG/trailblazing-a-safe-path-forward-e02f5f9ef0cc
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE<mailto:systemsafety at TechFak.Uni-Bielefeld.DE>
> Manage your subscription:
> https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
>

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE<mailto:systemsafety at TechFak.Uni-Bielefeld.DE>
Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
This email and its attachments may contain confidential and/or privileged information.  If you have received them in error you must not use, copy or disclose their content to any person.  Please notify the sender immediately and then delete this email from your system.  This e-mail has been scanned for viruses, but it is the responsibility of the recipient to conduct their own security measures. Airbus Operations Limited is not liable for any loss or damage arising from the receipt or use of this e-mail.

Airbus Operations Limited, a company registered in England and Wales, registration number, 3468788.  Registered office:  Pegasus House, Aerospace Avenue, Filton, Bristol, BS34 7PA, UK.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20190723/4f770823/attachment.html>