[SystemSafety] What do we know about software reliability?

[Peter Bernard Ladkin]

On 2020-09-24 06:10 , Brendan Mahony wrote:
>
> Most of the activities associated with software reliability estimation would seem to be worthy, but
> do they really result in any ability to predict future behaviour (in the absence of mathematical
> analysis of algorithmic features)?

Of course behaviour can be predicted.

You presumably use a laptop or desktop computer daily. Certain programs/applications crash every so
often. There are regularities in your use of the machine, and if your app crashed once a day and
restarted fine, you would not expect it today, under the same kind of use that you usually put it
to, to crash 1000 times and be unable to restart it, unless some new software versions had been
installed, or some HW is kaput, or your usage really has changed.

When you get up in the morning and start doing your daily digital chores, you reasonably expect the
same behaviour from your machine as you got yesterday, or the day before, or the day before that,
except if you installed an update. If that doesn't happen, you go to your equipment shop and let
them run the manufacturer's hardware integrity test, or ask your IT department to give you a new
laptop with an image of your backup on it.

Even the people who complain here vociferously that statistics doesn't apply to software expect
their laptop with its internet-facing programs to more or less behave the same way today as it did
yesterday. And the reason for that is their prior experience with it over years. They are thereby
predicting digital-system behaviour successfully. They don't get up in the morning sick with worry
that they won't be able to get any work done because their document-preparation program destroys
secondary memory and sets fire to the workroom. Or if they accidentally misspell a certain word,
Doomsday will descend on us all. (Or maybe they do, but it's not an expectation that the rest of us
might consider rational.)

Summary: there are regularities in digital-system use that we can reasonably expect, based on our
prior experience. And, furthermore, we all know this and behave accordingly for the most part.

The scientific task is to put trustworthy numbers on that phenomenon. And this is the point at which
things get tricky. People say (and have said similar to my colleagues) "Windows works fine and
doesn't crash. Why can't we use it to implement our SIL 4 function?"

The answer of course is that you can. Providing that you let the programmed app run on its target
hardware in an accurately simulated environment with dependable instrumentation (failure detection)
long enough to be reasonably confident that it won't fail more often than the regulation specifies.

The trouble with that activity is that, even if you can show the environment is accurately
simulated, it has to run for a very long time for you to develop reasonable confidence; probably
more time than you have available (Butler/Finelli 1993, Littlewood/Strigini 1993).

So far, none of stateval sceptics on this list have volunteered to answer the practical question
which I asked on 2020-09-15 at 1758 UTC, and elaborated on on 2020-09-16 at 0736 UTC. (Note BST =
UTC+1 and MEST = UTC +2.)

> For example, is Bayesian prior style reasoning applicable to (very complex) software controlled systems? 

What on earth is "Bayesian prior style reasoning"? Bayesian reasoning is a means of refining your
best guess as to the probability of a phenomenon through your further experience of that phenomenon.

I know of no reasoning method whose validity is affected by the size or complexity of its subject,
although the practicality of applying such reasoning is obviously so affected: much as we may like
Hoare logic, it is hard to use it effectively for OSs with O(10^8) LOC.

I am not going to address your series of rhetorical questions, because they lack specificity. They
are all a bit like asking if Hoare logic applies to your iPhone.

PBL

Prof. Peter Bernard Ladkin, Bielefeld, Germany
Styelfy Bleibgsnd
Tel+msg +49 (0)521 880 7319  www.rvs-bi.de





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.techfak.uni-bielefeld.de/pipermail/systemsafety/attachments/20200924/3fd1cc87/attachment.sig>