[SystemSafety] Modelling and coding guidelines: "Unambiguous Graphical Representation"

[Peter Bernard Ladkin]

On 2016-03-16 00:58 , Les Chambers wrote:
> The issue of making a safety case with the argument of proven in use
> resonates with me 

Good. You've unfortunately missed the open-comment period on the final draft of the TS, though. For
info, the process has been ongoing since Spring 2013.

>  At the point of making
> the safety case we had roughly 200 control computers operating in the field
> for a year with no failures or safety incidents. The accumulated operating
> hours allowed us to use certain equations that looked fantastic on paper and
> were probably the most compelling element of the case. In my engineering
> black-and-white heart though, I know this was complete BS. 

The equations aren't the point. It is the conditions on the data that are the point.

How were the usage logs?

* Did they record every input to the system?
* Could you guarantee that only those inputs to the system, with the frequences with which they were
input, would occur in the future use?
* Could you guarantee that there had been no masked failures? And that there were no masked
dependencies?
* 2 x 10exp(6) perfect ophours (= 200 systems in a year's operation) is good enough for SIL 1 but
not SIL 2 for a continuously-operational system.

If you couldn't have done all of that (and more), then the TS would say that your SW is not (yet)
"proven in use".

> But your disclosure that:
> 'The IEC is about to publish a technical specification on the criteria to be
> fulfilled for a component to be considered "proven in use".'
> ... is of grave concern to me, 

> 1. The concept of "same" does not apply to a software product. No two
> software products are the "same", ever. 

It certainly - and obviously - does apply.

If two instantiations are not identical then you can't lump them together for the purpose of joint
evaluation. People have been clear on that for four decades.

> 2. The concept of "identical environment" does not apply to software. As
> with item 1 there is no such thing. In the software context, there is no
> human verifiable criteria you could apply to defining one environment as
> identical to another.

Sure there is. The SW receives inputs. You can know what those parameters are. You better know what
those parameters are. If you don't know what they are, you have other problems than trying to assure
the SW for your system component as "proven in use". For example, you couldn't perform an adequate
hazard analysis.

> What is the IEC thinking! If what they're doing involves software they are
> formalising and promulgating utter BS. 

First, let me refer you to the current edition of IEC 61508, Parts 2 and 3. The conditions on
"proven in use" for SW are to my mind incoherent. So I would agree with you that, as things stand,
IEC 61508 is currently promoting an inappropriate approach to this.

It will continue to do so until the TS is published. The point of the TS is to try to fix this
anomaly by giving more appropriate guidance for SW.

However, I'm not sure it's appropriate to describe IEC as "thinking". It might be worth a couple of
words as to how it functions. It is a facilitating organisation which provides and supports a
process of turning a project into an international standard. Anyone can propose a standardisation
project. If it meets certain preliminary tests, then it will be proposed by a national standards
organisation (that of the originator) to the IEC, a call goes out from the IEC to national standards
organisations worldwide, and then from those national standards organisations to engineers
generally, for volunteers to participate on the project. Those people are often designated by their
organisations who have an interest in a resulting standard. Sometimes, as in my case, we just do it
because we think (or hope!) it is a valuable thing to do.

In this case, someone thought it would be a good idea to have coherent guidance on evaluating
safety-critical SW as "proven in use", and proposed it to the IEC. I can actually name names.
Indeed, four people on this list are in the IEC working group.

This is not the first time the subject of evaluation of existing SW has been discussed here. It
would have been appropriate for comments to be directed to the IEC at an earlier point. All IEC
projects have to address each submitted comment. The comments process is pretty bureaucratic, but it
is designed in part to weed out non-serious commentary and to direct commentary specifically towards
changing an existing document: you have to specify exactly which paragraphs or sentences in the
document are unsatisfactory, why they are unsatisfactory, and explicitly suggest a substitute,
including omission if appropriate.

This is common to the ISO, IEC, CENELEC. There are people here who think the standards process is
broken; indeed I think it is broken in certain respects, but I think the commentary aspect works
moderately well, even though there is plenty of room for improvement. A recent revision of a
European standard for safety-critical digital systems on railways was abandoned, because the draft
garnered 5,000 comments or so, and they couldn't be worked through by the committee in the time
period available for completion of the project. The project had to be formally abandoned. That is
how the fact of no consensus translates into a result.

Two main failings of the standardisation process from my point of view are that:

* Not everyone who claims they wish to comment is aware of how, or is enabled, to do so. Anyone can
download the comments form from the IEC (it is unfortunately bound to one proprietary
document-processing SW and often gives poor results when used with others. An argument for a
standard, if ever there was one). However, you have to pass your comments to the responsible
national committee, who processes them and if necessary forwards them to the IEC working group. It
is thought by people "inside" the process that this is straightforward. However, it can be
frustratingly hard for people not otherwise involved to engage in this process. Much of that is down
to how well the national committees and their infrastructure function.

* Some working groups can be spectacularly inept and there is no final technical quality control in
the process. To my mind, and in my experience, independent technical reviews of proposed standards
documents would do wonders for the technical quality of some standards. One can imagine that, for
example, IEC 61508 would have ended up being a fifth of the size it is, and better for it.

> With this kind of Stone Age thinking they should go back to standardising
> nuts and bolts.

People have accused me of all sorts of mental misdemeanors throughout the years, but "Stone Age
thinking" is genuinely new. For the record, my leopard skins are all faux but my knotty club is real
enough.

PBL

Prof. Peter Bernard Ladkin, Faculty of Technology, University of Bielefeld, 33594 Bielefeld, Germany
Je suis Charlie
Tel+msg +49 (0)521 880 7319  www.rvs.uni-bielefeld.de





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: OpenPGP digital signature
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20160316/887a453a/attachment.pgp>