[SystemSafety] What do we know about software reliability?

[Peter Bishop]

On 15/09/2020 11:28, Peter Bernard Ladkin wrote:
>
> On 2020-09-15 11:21 , Coq, Thierry wrote:
>> ...Ariane A501 flight has disproved any probabilistic approach on software that would not
>> start with a frequency of failure of 1. 

I have to disagree. It does not disprove the probabilistic approach. The
point of the tests is to set an upper bound on the likelihood of failure.

Statistical test should use a realistic profile for the tests. So for
Ariane V, if you ran the system in an iron bird simulator with a Ariane
V launch profile, it would indeed crash first time - so the bound is
effectively 1 given the test result.

>From this we would conclude it is highly unlikely to have a crash
probability better than 10-2 (or some such desired target).

If we fixed the bug and and then ran 300 simulated launches without
failure we conclude it would meet the target with 95% confidence.

It does not mean the system is bug-free, just that any bugs that may be
present are unlikely to be activated for a realistic profile.

If you fixed the original bug and and did not do the 300 tests, what
confidence would you have that the next launch of the Ariane V would
succeed?
- might be cheaper to do the tests rather than finding out the hard way.

Peter

> I have no idea what this sentence means.
>
> I think everybody would agree that the environmental circumstances leading to failure of the Ariane
> FLight 501 control system were certain to arise during launch. I don't see what probability has to
> do with it.
>
> Whatever people's inclinations to statistical evaluations of SW, we are about to be deluged by it.
> The DLNN assistance functions in the automation of road vehicles can at present only be justified by
> an assessment of their in-service behaviour. There are going to be numerical requirements on their
> dependability and vendors will be providing argument that those numerical requirements are fulfilled.
>
> The statisticians know well how much evidence has to be produced in order to derive conclusions on
> reliability to an appropriate level of confidence. (See, for example, Peter Bishop's comment here on
> the Tempe accident.) It is a lot higher than what, as far as I can tell, the vendors of such
> equipment are likely be able to produce. What's going to happen?
>
> PBL
>
> Prof. Peter Bernard Ladkin, Bielefeld, Germany
> Styelfy Bleibgsnd
> Tel+msg +49 (0)521 880 7319  www.rvs-bi.de
>
>
>
>
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety

-- 

Peter Bishop
Chief Scientist
Adelard LLP
24 Waterside, 44-48 Wharf Road, London N1 7UX

Email: pgb at adelard.com
Tel:  +44-(0)20-7832 5850

Registered office: 5th Floor, Ashford Commercial Quarter, 1 Dover Place, Ashford, Kent TN23 1FB
Registered in England & Wales no. OC 304551. VAT no. 454 489808

This e-mail, and any attachments, is confidential and for the use of
the addressee only. If you are not the intended recipient, please
telephone 020 7832 5850. We do not accept legal responsibility for
this e-mail or any viruses.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/pipermail/systemsafety/attachments/20200915/7399a0a0/attachment.html>