[SystemSafety] Uber Advanced Technologies Group publishes its "Safety Case Framework"

[Olwen Morgan]

On 23/07/2019 14:01, M A Jackson wrote:
> Control over the fixed infrastructure for AVs is a stronger constraint than is necessary in principle.
>
> Design of any cyber-physical system and its proposed functions and behaviours must rely on assumptions about its operating environment in space and time. In principle, these assumptions must be explicitly articulated and analysed, and assessed for the risk that they may not hold. To give a topical illustration: the Apollo 11 designers had no control over the relevant environment; but they had sufficiently reliable knowledge of the relevant environmental properties and behaviours that could be assumed in the small region of the solar system in which the moonshot would take place.
>
> Cars driven by people can cope with successfully with a wide environmental range—let’s call it E1. For a proposed AV design an explicit statement of the assumed environment E2 (presumably a subset of E1) is a sine qua non. Without this explicit statement announcements like Uber’s ATG Framework have little meaning. Control over the fixed infrastructure certainly makes this explicit statement very much easier. Another approach is strict geofencing in a specialised area. For example, a  low-speed AV taxi service has been provided for a retirement community occupying a gated space of a few square miles, where the only road users are pedestrians, golf buggies, and the AV taxis.
>
> — Michael Jackson
>
I'm not convinced that the local universe analogy is actually apt here. 
I agree that the constraint is *in principle* stronger than needed. On 
the other hand, I think it likely that *in practice* AVs will produce a 
steady stream of fatalities unless they are operating within some kind 
of controlled infrastructure.

I'm not sure about geofencing. It may (or may not) prevent AVs from 
going where they shouldn't when everything is working OK but I'm 
pessimistic about the prospects for fail-safe design in this area. If we 
could develop cheap standardised physical fencing (e.g. by producing 
barriers from recycled plastic), then I'd have a bit more confidence 
about safer operation.

One thing I think you are right about is being explicit about the 
proposed operating environment. If you cannot define a safe operating 
envelope as a subset of a wider phase space, then all bets are off as 
regards achieving a safe design.


Olwen