[SystemSafety] Does "reliable" mean "safe" and or "secure" or neither?

[Peter Bernard Ladkin]

First, a response to Michael (without quoting him). I accept that there is a dynamic with expressing
views on this list which not everyone feels comfortable negotiating. But it's not just a matter of
worrying about being "shot down" by another list member. Whatever is said here is publicly archived,
permanently. Everyone in the world with an Internet connection can read it. You're in a glass bowl.

Whatever people's individual views on matters such as statistical software reliability estimating,
there is a large collection of material on this list in the last sixteen months which includes
material which you will not find elsewhere. Various people's 200-word views on the matter, for
example. It is also clear that some people use all the resources at their disposal to construct the
best arguments for their view, in dialogue, and that happens on all sides of a question. I think the
value of such discussions is incomparable.

Now back to tech.

On 2016-04-24 09:09 , Coq, Thierry wrote:
> ..... this last exchange seems to me a debate on authority.
> On our left, we have DO-178. B now C.
> On our right, we have IEC, IEEE, Musa, etc.

There are actually two issues here. One is whether the notion of software reliability makes sense. I
think it clearly does. I show below that ED-12C implicitly acknowledges that it does, contrary to
what Nick Tudor may be hinting.

Second is whether the notion of software reliability is useful. For some industries it currently is
(British nuclear power, for example). For some industries it currently is not (civil aeronautics,
for example). And many industries don't know (German railways, for example).

The reason why software reliability considered as a collection of methods is not useful in civil
aeronautics was clearly set out by one (actually, two) of the very people Nick is arguing with,
namely Bev, along with Lorenzo Strigini, 23 years ago in a seminal paper. Those considerations, like
most math, have not changed. Since then, some methods have appeared which allow the inference of
ultrahigh reliability from feasible evidence - one of them due to Bev - but they are currently
limited to very specific architectures. For a more recent summary of why software reliability
methods are not current useful in civil aeronautics, I recommend people talk to Mike Holloway, who
is eloquent on the issue.

The reason why software reliability considered as a collection of methods is useful for the British
nuclear industry is that there are some procedures in nuclear power plants which are invoked
(demanded) less than once a year, but which really need to work when they are invoked. Much lower
levels of reliability are required, because you get 10^4 for free (number of hours in a year) when
you are figuring out any likelihood of failure over the course of a system lifetime. The methods
work for that application.

Concerning rail applications, senior railway engineers in Germany believe that they have methods
which work for some key rail applications. They will be presenting their method at a Safety
Enfgineering symposium in Cologne on May 10. Before that, I understand one of them will be
presenting them on Wednesday morning at the safe.tech colloquium in Munich.

> To go further, it is plain fact that the aeronautics industry has demonstrated it doesn't need "software reliability" to 
> deliver highly reliable automated systems, or systems of systems.

Yes. Systems which each cost eight- or nine-digit sums of money to buy, and which have to be sold in
their near-thousands to recover development costs. That is unique, and is hardly a model for any
other industry.

Now to say exactly what ED-12C has to say on software reliability.

The word "reliability" occurs five times in ED-12C. One is in the Contents page (the title of
Section 12.3.3). Once is general (p5, Section 2.1 intro). The phrase "software reliability" occurs
three times.

The introduction to Section 2.3 says inter alia

[begin citation]
Development of software to a software level does not imply the assignment of a failure rate for that
software. Thus, software reliability rates based on software levels cannot be used by the system
safety assessment process in the same way as hardware failure rates.
[end citation]

There there is a section on "software reliability models". Here it is.

[begin citation]
Section 12.3.3. Software Reliability Models

Many methods for predicting software reliability based on developmental metrics have been published,
for example, software structure, defect detection rate, etc. This document does not provide guidance
for those types of methods, because at the time of writing, currently available methods did not
provide results in which confidence can be placed.
[end citation]

It should surely be apparent that ED-12C (= DO-178C) says at no point that there is no such thing as
software reliability. It should be apparent that it rather acknowledges that there is. It uses the
phrase twice to say things about how they are to be considered in civil aeronautical system
assessment (namely, not applicable to us).

PBL

Prof. Peter Bernard Ladkin, Faculty of Technology, University of Bielefeld, 33594 Bielefeld, Germany
Je suis Charlie
Tel+msg +49 (0)521 880 7319  www.rvs.uni-bielefeld.de





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: OpenPGP digital signature
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20160424/66516ce1/attachment.pgp>