[SystemSafety] Software reliability (or whatever you would prefer to call it)

David Ward david.ward at mira.co.uk
Fri Mar 6 12:02:37 CET 2015


Michael's proposal is very similar to the approach contained in ISO 26262 Part 8 Clause 12 "Qualification of software components".

To grossly simplify, it requires generation of confidence through black-box testing (including considering abnormal cases as well as testing against requirements) but for ASIL D evidence of structural coverage is also required.


David Ward (Dr)
Head of Functional Safety
MIRA Ltd

-----Original Message-----
From: systemsafety-bounces at lists.techfak.uni-bielefeld.de [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Michael J. Pont
Sent: 06 March 2015 10:56
To: systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] Software reliability (or whatever you would prefer to call it)

Martyn suggests that we put the language to one side.

My take on the core problem.

IEC 61508-7 [2010] Annex D "provides initial guidelines on the use of a probabilistic approach to determining software safety integrity for pre-developed software based on operational experience. This approach is considered particularly appropriate as part of the qualification of operating systems, library modules, compilers and other system software."

In effect, I select an appropriate set of test data, run my system for a long time (or run lots of systems for a short time) and conclude - if no failures are detected - that the system is safe.  

The longer that I test for, the higher the SIL level that can be assigned to the component that is being evaluated.

In my book, this is Black-Box testing.

If we revise this appendix as Peter proposes, then we may be able to help people to select more appropriate test data (and this may be an improvement)
- but this will still be Black Box testing.

If we can't avoid this appendix altogether (and I'm sure that Bertrand is right about this), then we should - surely - be able to require some additional "White Box" assessments, such as code reviews, design reviews, etc (in line with the rest of the standard).  

If we can achieve this, I would sleep more easily.

Michael.

-----Original Message-----
From: systemsafety-bounces at lists.techfak.uni-bielefeld.de
[mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Martyn Thomas
Sent: 06 March 2015 09:56
To: systemsafety at lists.techfak.uni-bielefeld.de
Subject: [SystemSafety] Software reliability (or whatever you would prefer to call it)

I'm puzzled by much of this discussion. Consider this common example:

A company creates a software package and submits it for beta testing by a group of users. Assume that the package reports how often it is used and for how long, and the users report all errors they encounter. Assume there is a single instance of the software on a server that all the users use.

The company corrects some of the errors that are reported.

The company calculates some measure of the amount of usage before failure.
Call it MTBF.

The MTBF is observed to increase.

What word shall we use to describe the property of the software that is increasing?

I'd call it "reliability". If you would, too, then how can software reliability not exist?

I don't mind if you want to use a different word to describe the property.
Let's just agree one, do a global replace in the offending standards and move on ...

... to discussing a practical upper bound on the "reliability" that can be assessed in this way - and on the assumptions that should be made explicit before using any such assessment as a prediction of future performance.

Martyn







_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE

--
MIRA Ltd

Watling Street, Nuneaton, Warwickshire, CV10 0TU, England
Registered in England and Wales No. 402570
VAT Registration  GB 100 1464 84

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.


More information about the systemsafety mailing list