[SystemSafety] What do we know about software reliability?

[Martyn Thomas]

Why are you completely dismissing software reliablity?

Is it not the case that if you can tolerate a failure rate of once in
1000 hours, 99% confidence through testing would take about 200 days to
demonstrate (so long as the test environment is "sufficiently" like the
future operating environment and you are able to detaect every failure
correctly)?

Of course, the testing would have to be repeated following a change to
the software, unless you have enough formality to show that the change
cannot affect reliability.

In specific circumstances, you can do better than this. Bev Littlewood's
published papers provide strong evidence and a rich bibliography. Bev's
paper on "How reliable is a program that has never failed?" offers a
useful rule-of-thumb: that aften n hours of fault free operation, there
is about 50% chance of a failure in the following n hours (subject to
some obvious constraints).

The difficulties rapidly escalate when you need 10^-4 or better at >90%
confidence.

Martyn

On 14/09/2020 14:14, SPRIGGS, John J wrote:
>
> In my experience, if Software Reliability is mentioned at a
> conference, at least one member of the audience will laugh, and if it
> is mentioned in a work discussion, at least one member of the group
> will get angry.
>
> Interestingly, some of the same people who say it is impossible to
> quantify software failure rates will set numerical requirements for
> Software Availability – if you get one of those, ask the Customer how
> (s)he wants you to demonstrate satisfaction of the requirement.
>
>  
>
> John
>
> *From:*systemsafety
> <systemsafety-bounces at lists.techfak.uni-bielefeld.de> *On Behalf Of
> *Derek M Jones
> *Sent:* 14 September 2020 12:54
> *To:* systemsafety at lists.techfak.uni-bielefeld.de
> *Subject:* [SystemSafety] What do we know about software reliability?
>
>  
>
> All,
>
> What do we know about software reliability?
>
> The answer appears to be, not a lot:
> http://shape-of-code.coding-guidelines.com/2020/09/13/learning-useful-stuff-from-the-reliability-chapter-of-my-book/
> <http://shape-of-code.coding-guidelines.com/2020/09/13/learning-useful-stuff-from-the-reliability-chapter-of-my-book>
>
> -- 
> Derek M. Jones Evidence-based software engineering
> tel: +44 (0)1252 520667 blog:shape-of-code.coding-guidelines.com
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> <mailto:systemsafety at TechFak.Uni-Bielefeld.DE>
> Manage your subscription:
> https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
>
>
>
> ------------------------------------------------------------------------
> If you are not the intended recipient, please notify our Help Desk at
> Email Information.Solutions at nats.co.uk immediately. You should not
> copy or use this email or attachment(s) for any purpose nor disclose
> their contents to any other person.
>
> NATS computer systems may be monitored and communications carried on
> them recorded, to secure the effective operation of the system.
>
> Please note that neither NATS nor the sender accepts any
> responsibility for viruses or any losses caused as a result of viruses
> and it is your responsibility to scan or otherwise check this email
> and any attachments.
>
> NATS means NATS (En Route) plc (company number: 4129273), NATS
> (Services) Ltd (company number 4129270), NATSNAV Ltd (company number:
> 4164590) or NATS Ltd (company number 3155567) or NATS Holdings Ltd
> (company number 4138218). All companies are registered in England and
> their registered office is at 4000 Parkway, Whiteley, Fareham,
> Hampshire, PO15 7FL.
> ------------------------------------------------------------------------
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/pipermail/systemsafety/attachments/20200914/f7a50889/attachment-0001.html>