[SystemSafety] Software Safety Assessment

Thu Jul 9 06:31:36 CEST 2015

Carl,

My perspective is as follows. In summary I don't think your question or the
answers have much to do with 'safety', per se, but a lot to do with role of
formalism in engineering.

1. Is it acceptable to use an obsolete (but still valid) safety standard to
assess new software?

A: Yes, unless there exist latter standards that specifically identify and
fix inadequacies in the former, thereby invalidating it. See for example
the issue of DO-178C, which resolved a number of recognized problems with
its predecessor 178B.

Remember that complying to a standard is not an argument about whether
you're safe per se but about compliance with a defined protocol. It's
similar to the principle of legal formalism whereby the law is deemed to
define itself by rigorous adherence to the statutes even (and perhaps
especially) when it cuts against common sense. This is one view of the law
by the way.

As to how close the standard gets to safety or the law gets to justice, who
knows. But from a formalism perspective we're not arguing that are we?

2. Is the SIL1 claim for 10 year old Project A invalid because the
checklist could have been better?

A: No, from the same perspective, if it's part of the same standard and
that has not been explicitly superseded by another. So it's a perfectly
valid formalism, go to it.

3. If Project B used the old checklist from Project A would that be
adequate?

A: From this perspective, yes as long as there was no superseding standard
that obsoleted it for that reason (e.g poor checklists).

>From a practical perspective, now that you know there's a problem?

Maybe... But do you? Really?

Matthew Squair

MIEAust, CPEng
Mob: +61 488770655
Email; Mattsquair at gmail.com
Web: http://criticaluncertainties.com

On 8 Jul 2015, at 10:40 pm, Carl Sandom <carl at isys-integrity.com> wrote:

  It's complicated and I was trying to avoid too much detail to get to the
central questions.

It has been 'fielded' and is being 'used' during extended V&V activities
(in parallel with the old system) but it is not yet considered fully
operational. Safety assessment of some software aspects continues on
Program A but not the 'process-based' software development assessment which
was the subject of Standard X and the original checklist in 2004. For the
scenario, take it as read that Standard X tools and techniques *are still
valid* even though it is now obsolete.

My original questions slightly modified are:

1. Is it acceptable to use an obsolete (but still valid) safety standard to
assess new software?

2. Is the SIL1 claim for 10 year old Project A invalid because the
checklist could have been better?

3. If Project B used the old checklist from Project A would that be
adequate?

Cheers

Carl

*From:* Drew Rae [mailto:d.rae at griffith.edu.au <d.rae at griffith.edu.au>]
*Sent:* 08 July 2015 13:15
*To:* Carl Sandom
*Subject:* Re: [SystemSafety] Software Safety Assessment

Carl,

You may need to clarify here.

The software was assessed 10 years ago.

The project has not yet been fielded.

There must be a missing detail for this not to be 10 year old unused code.
Has it been used in some context other than Project A during that time?

* This message is from my work email

* I can also be contacted on andrew at ajrae.com

* My mobile number is 0450 161 361

* My desk phone is 07 37359764

* My safety podcast is DisasterCast.co.uk

On 08/07/2015, at 10:08 PM, Carl Sandom wrote:

  Some important clarifications:

Project A has not yet been fielded but the software was assessed against
Standard X 10 years ago.

The techniques applied to Project A were appropriate and fulfilled the
requirements of Standard X......at that time and now. But the evidence from
the checklist could have been better.

No idea why you assumed unused 10 year old code but that's not the case
here.

Cheers

Carl

*From:* Drew Rae [mailto:d.rae at griffith.edu.au <d.rae at griffith.edu.au>]
*Sent:* 08 July 2015 12:57
*To:* Carl Sandom
*Cc:* systemsafety at lists.techfak.uni-bielefeld.de
*Subject:* Re: [SystemSafety] Software Safety Assessment

"Acceptable" either comes from some form of social consensus, or from a
belief that the particular techniques applied are appropriate for that
particular piece of software. The way you've phrased the question, it
sounds like there is significant doubt that the techniques applied on
Project A were appropriate for Project A. If Project A hasn't been
previously deployed, that's like saying

"I've got this piece of old flex cable sitting under the house. It doesn't
meet current electrical safety standards - in fact it wouldn't have met 10
year old safety standards except they were a bit vague and there was a
loophole - but I should be allowed to use it anyway. And since I'm using
dodgy flex anyway, you don't mind if I apply the same standards to my new
wiring as well, do you?".

Compliance with a standard is typically the _minimum_ required for safety.
Safety requires compliance, but compliance doesn't give safety. If there's
doubt that the checklist for Project A was good enough, no amount of
weaselling about standards is going to make it good enough.

As others have said though, if you just want acceptability as a social
consensus, then it's not a question that can be answered in the abstract,
only in terms of the supplier, customer, and any contract or regulator
involved.

Incidentally - someone is resurrecting 10 year old code that's been sitting
unused, and significantly hacking it around, and they intend to use it for
a safety application? And that's not enough to make people run screaming
for cover? I can understand a need to modify legacy embedded code that's
been in use, but unused 10 year old code?

* This message is from my work email

* I can also be contacted on andrew at ajrae.com

* My mobile number is 0450 161 361

* My desk phone is 07 37359764

* My safety podcast is DisasterCast.co.uk

On 08/07/2015, at 7:53 PM, Carl Sandom wrote:

   Consider the following scenario:

In 2004 Project A software was assessed against a safety standard (let's
call it Standard X). Standard X had a very prescriptive list of software
safety requirements and a simple checklist was used for assessing SIL1
compliance.

In 2014, Project B began to integrate significant new functionality into
Project A. Standard X, which was by 2014 an obsolete standard, was used to
assess the significantly smaller software baseline of Project B. Under
modern scrutiny, the simple Standard X checklist used for Project A in 2004
was not as explicit as it could have been and it was decided to use an
improved checklist for Project B.

A couple of important questions can be raised with this scenario:

1. Is it acceptable to use an obsolete safety standard to assess software?

2. Is the SIL1 claim for 10 year old Project A invalid because the
checklist could have been better?

3. If Project B used the old checklist from Project A would that be
adequate?

I've been having some interesting discussions with the Project Managers
involved, any thoughts?

Regards

Carl

_________________________________

Dr. Carl Sandom CErgHF CEng PhD

Director

iSys Integrity Ltd.

+44 7967 672560

carl at isys-integrity.com

www.isys-integrity.com

_________________________________

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20150709/e2714f6b/attachment-0001.html>