[SystemSafety] Safety Culture redux (David Green)

[Peter Bernard Ladkin]

On 2018-02-22 00:54 , Steve Tockey wrote:
> 
> IEEE already has a recommended vocabulary:
> 
> Incident = any difference between the observed result and the expected result
> 
> Failure = it has been determined that the observed result is incorrect 
> 
> Fault or Defect = the aspect of the code caused the incorrect result
> 
> 
> If adequate vocabulary already exists, why try to invent new terms?
Because there are things wrong with this series of definitions.

First, an incident in most people's usage is an event. With nothing counterfactual about it. It just
is (or was). A "difference" is not an event, but a contrastive feature of two things, one of which
is counterfactual. So the definition of "incident" here confuses an event (what did happen) with its
features (that one of the aspects contrasts with what was expected).

Contrastive description is common and useful, but it is better not to conflate an event with its
description, for the following reasons amongst others. Obviously, in order to individuate an event
you do so with a description, because that is in part how language works. A description (if it fits)
picks out an aspect of an event. But that aspect may be superficial, and not key. If someone
proffers a superficial description, you want a second person to be able to say "that is not all of
what went wrong here, that is just a part of it". Whereas, with this definition, the second person
is not refining what the first said by identifying a more significant aspect, they are literally
describing a different incident. You have as many different incidents as you do aspects, and the set
of aspects is not usually very well bounded. William of Ockham had something to say about that.

It is usual to designate a complex-system incident or accident that as an event, one event. But
according to the IEEE definition, it becomes a plethora of difference specifications.

Second, the definition of "failure" requires a "determination", which is a human act. If the system
is not sociotechnical, then failure is an objective matter without a social component. Further, I
think we can bet that the IEEE does not say what a "determination" consists in. Continuing, the
definition makes essential use of the notion of "correct". Is that defined somewhere? "Correct" and
"incorrect" are both notions which involve a comparison between a result and a norm. What norm would
that be? "What the system should have done"? The problem there is the word "should", which has a
moral connotation. "What person X thinks would have been a more appropriate outcome"? How do you
pick person X? "What most people dealing with the system agree would have been a more appropriate
outcome"? How do you select that crowd? We might like to say "What the system specification says
happens in that case". But that supposes the system has a specification, and that specification is
adequate to determine how the system behaves in this case. One suspects the definition was
formulated to finesse that need.

Third, the common idea of "fault" is "<certain system aspects> which caused the failure" (with
"<certain system aspects>" to be determined. It was likely causally contributory to the failure that
the system received certain inputs - the definition of "fault" here entails that the presence of
those inputs are part of the fault. Should that be so? Intuitively, we would say no: (a) if the
inputs were inappropriate, they should have been filtered and the lack of filtering was part of the
fault, not the inputs themselves; (b) if the inputs were appropriate, then it is the way the system
processed them that is usually taken to be the fault, not the inputs themselves.

Can we fix these issues easily? Sure. I recommend, as usual, the definitions in
https://causalis.com/90-publications/99-downloads/DefinitionsForSafetyEngineering.pdf

PBL

Prof. Peter Bernard Ladkin, Bielefeld, Germany
MoreInCommon
Je suis Charlie
Tel+msg +49 (0)521 880 7319  www.rvs-bi.de





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 525 bytes
Desc: OpenPGP digital signature
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180222/42102196/attachment.sig>