[SystemSafety] Software Safety Assessment

GRAYDON, PATRICK J. (LARC-D320) patrick.j.graydon at nasa.gov
Wed Jul 8 15:26:24 CEST 2015


When you say that the obsolete standard is “still valid”, what do you mean?

To me, that phrase implies that you have some reason to believe that conformance to the old standard implies that the resulting product is adequately safe to operate as intended.  (Or perhaps adequately likely to be correct, depending on what standards conformance is meant to tell you.  I’ll write the rest of this assuming that safety is the goal, but my question probably applies to other goals as well.) 

If we had a trustworthy means of determining whether conformance to a given standard implies adequate safety — and I am not aware of any — then the issue of obsolescence is irrelevant: apply the means, determine if the standard you want to use implies adequate safety in the circumstances, and, if so, use it.  (It might be ‘obsolete’ because it does not discuss some new development technique that you aren’t using anyway.)

If we don’t have a trustworthy means of making this determination, then doesn’t the question of which standard to use come down to (a) which can be applied feasibly in the application in question and (b) whether we have a good reason to think the standard doesn’t imply adequate safety?  And, if that is the case, then doesn’t your question really turn on whether the reasons for obsoleting the standard and the things in the checklist that “could have been better” constitute evidence that it is unacceptably likely that a system of the kind in question could conform and yet be unsafe?

Kind regards,
— Patrick

<Insert the usual disclaimer about not being a spokesperson for my employer here.>


On 2015-07-08, at 08:36, Carl Sandom <carl at isys-integrity.com> wrote:

> It's complicated and I was trying to avoid too much detail to get to the central questions.
> 
> It has been 'fielded' and is being 'used' during extended V&V activities (in parallel with the old system) but it is not yet considered fully operational. Safety assessment of some software aspects continues on Program A but not the 'process-based' software development assessment which was the subject of Standard X and the original checklist in 2004. For the scenario, take it as read that Standard X tools and techniques are still valid even though it is now obsolete.
> 
> My original questions slightly modified are:
> 
> 1. Is it acceptable to use an obsolete (but still valid) safety standard to assess new software?
> 
> 2. Is the SIL1 claim for 10 year old Project A invalid because the checklist could have been better?
> 
> 3. If Project B used the old checklist from Project A would that be adequate?
> 
> Cheers
> Carl
> 
> From: Drew Rae [mailto:d.rae at griffith.edu.au] 
> Sent: 08 July 2015 13:15
> To: Carl Sandom
> Subject: Re: [SystemSafety] Software Safety Assessment
> 
> Carl,
> You may need to clarify here. 
> 
> The software was assessed 10 years ago. 
> The project has not yet been fielded. 
> 
> There must be a missing detail for this not to be 10 year old unused code. Has it been used in some context other than Project A during that time? 
> 
> * This message is from my work email
> * I can also be contacted on andrew at ajrae.com
> * My mobile number is 0450 161 361
> * My desk phone is 07 37359764
> * My safety podcast is DisasterCast.co.uk
> 
> 
> 
> 
> 
> On 08/07/2015, at 10:08 PM, Carl Sandom wrote:
> 
> 
> Some important clarifications:
> 
> Project A has not yet been fielded but the software was assessed against Standard X 10 years ago.
> 
> The techniques applied to Project A were appropriate and fulfilled the requirements of Standard X......at that time and now. But the evidence from the checklist could have been better.
> 
> No idea why you assumed unused 10 year old code but that's not the case here.
> 
> Cheers
> Carl
> 
> From: Drew Rae [mailto:d.rae at griffith.edu.au] 
> Sent: 08 July 2015 12:57
> To: Carl Sandom
> Cc: systemsafety at lists.techfak.uni-bielefeld.de
> Subject: Re: [SystemSafety] Software Safety Assessment
> 
> "Acceptable" either comes from some form of social consensus, or from a belief that the particular techniques applied are appropriate for that particular piece of software. The way you've phrased the question, it sounds like there is significant doubt that the techniques applied on Project A were appropriate for Project A. If Project A hasn't been previously deployed, that's like saying 
> 
> "I've got this piece of old flex cable sitting under the house. It doesn't meet current electrical safety standards - in fact it wouldn't have met 10 year old safety standards except they were a bit vague and there was a loophole - but I should be allowed to use it anyway. And since I'm using dodgy flex anyway, you don't mind if I apply the same standards to my new wiring as well, do you?". 
> 
> Compliance with a standard is typically the _minimum_ required for safety. Safety requires compliance, but compliance doesn't give safety. If there's doubt that the checklist for Project A was good enough, no amount of weaselling about standards is going to make it good enough. 
> 
> As others have said though, if you just want acceptability as a social consensus, then it's not a question that can be answered in the abstract, only in terms of the supplier, customer, and any contract or regulator involved. 
> 
> Incidentally - someone is resurrecting 10 year old code that's been sitting unused, and significantly hacking it around, and they intend to use it for a safety application? And that's not enough to make people run screaming for cover? I can understand a need to modify legacy embedded code that's been in use, but unused 10 year old code?
> 
> 
> * This message is from my work email
> * I can also be contacted on andrew at ajrae.com
> * My mobile number is 0450 161 361
> * My desk phone is 07 37359764
> * My safety podcast is DisasterCast.co.uk
> 
> 
> 
> 
> 
> 
> On 08/07/2015, at 7:53 PM, Carl Sandom wrote:
> 
> 
> 
> Consider the following scenario:
> 
> In 2004 Project A software was assessed against a safety standard (let's call it Standard X). Standard X had a very prescriptive list of software safety requirements and a simple checklist was used for assessing SIL1 compliance.
> 
> In 2014, Project B began to integrate significant new functionality into Project A. Standard X, which was by 2014 an obsolete standard, was used to assess the significantly smaller software baseline of Project B. Under modern scrutiny, the simple Standard X checklist used for Project A in 2004 was not as explicit as it could have been and it was decided to use an improved checklist for Project B.
> 
> A couple of important questions can be raised with this scenario:
> 
> 1. Is it acceptable to use an obsolete safety standard to assess software?
> 
> 2. Is the SIL1 claim for 10 year old Project A invalid because the checklist could have been better?
> 
> 3. If Project B used the old checklist from Project A would that be adequate?
> 
> I've been having some interesting discussions with the Project Managers involved, any thoughts?
> 
> Regards
> Carl
> _________________________________
> Dr. Carl Sandom CErgHF CEng PhD
> Director
> iSys Integrity Ltd.
> +44 7967 672560
> carl at isys-integrity.com
> www.isys-integrity.com
> _________________________________
> 
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> 
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> 
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 12032 bytes
Desc: not available
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20150708/6c5a52ba/attachment.bin>


More information about the systemsafety mailing list