[SystemSafety] Elephants, dinosaurs and integrating the VLA model

[Prof. Dr. Peter Bernard Ladkin]

On 2023-08-04 07:37 , Steve Tockey wrote:
> ... the Elaine Herzberg case. .... It never needed to have happened in the first place. There was never any trade-off to be made here.

It has been pointed out that Herzberg would likely have been detected by the original Volvo 
object-detection system implemented in the vehicle before it was modified by Uber. This system was 
reportedly disabled by Uber in favor of its own design.

I agree with Steve that any high-level hazard analysis of the Uber design should have flagged up the 
issue of de-reifying unclassified objects. A hazard and risk analysis (HRA) is required in any ISO 
or IEC standard which has safety aspects; this requirement is specified in ISO/IEC Guide 51, where 
it is also specified what an HRA is and what the terms mean.

Both ISO 26262 and IEC 61508 necessarily conform to the Guide 51 requirement: both require HRAs, not 
only of the high-level system design but also at subsystem level. There is thus an argument that the 
Uber design did not conform with ISO 26262.

(I've indicated that I don't see where Les is going, but the reasoning above refutes any suggestion 
that existing standards don't have that kind of activity covered.)

The issue of ineffective HRA is one to which some of us have devoted part of our working lives. 
Prominent instances are the Tempe fatal accident, the Boeing 737 MAX accidents, and the UK RAF 
Nimrod accident.

The Haddon-Cave inquiry into Nimrod underlined the box-checking nature of the performed safety case 
and its failure to flag the dangers of the design of the refueling system, obvious even to the 
non-engineer Haddon-Cave.

HazAn for commercial aircraft systems is explicitly required by 14 CFR §25.1309(d). The 
Congressional inquiry into the Boeing 737 MAX development noted that the original MCAS design was 
for one iteration and limited activation time/travel/rate of the STS. It was this design on which 
the HazAn was performed. (I would still query whether this original analysis was right, but that's 
another story.) The MCAS design was modified to extend activation time and travel and rate and it 
was determined by Boeing that the HazAn did not need to be repeated. The MCAS on the production 
aircraft, though, allowed repeated activations. No HazAn had been performed on 
MCAS-with-repeated-activation. I am still flabbergasted that the aircraft had been certified with no 
valid HazAn of the implemented system.

PBL

Prof. i.R. Dr. Peter Bernard Ladkin, Bielefeld, Germany
Tel+msg +49 (0)521 880 7319  www.rvs-bi.de