[SystemSafety] Safety Case Standards and Experience

Mon Feb 17 00:42:45 CET 2014

Les,

Here are 2 domain specific standards which I have encountered in practice..

In the ATM-domain (Air Traffic Management) there is UK-standard named
CAP670, in particular you may find <Part B Section 3 Appendix B to SW01
“Argument and Evidence Concept”> of interest.

CAP670 is in the public domain, you can find it here :
http://www.caa.co.uk/docs/33/CAP670ISs03Amdt01.pdf

In the automotive domain, ISO 26262 is applicable which is an adaptation of
IEC-61508. ISO 26262 does not adopt the safety case concept in the sense of
a safety argument, backing evidence etc.

ISO26262 prescribes the development of a whole range of Work Products, this
collection of WPs is considered the safety case (evidence), it does not
however prescribe the development of a safety 

argument.

If safety activities are not adequately integrated in the project’s
lifecycle, then it is impossible to run an effective system safety program.
Adequate integration includes both the “clear” identification 

of safety activities and their relevant dependencies w.r.t. the engineering
activities. These dependencies should address both prerequisite engineering
information necessary to perform safety activities,

and the output of safety activities (safety requirements) which flow into
the engineering process. I find that strict safety case regimes are
particularly weak with respect to, among others, the integration 

of safety in the project’s lifecycle.

I find discussions about “safety case vs prescriptive approaches” very
interesting but I think that these can only be constructive if the term
“safety case” is given substance. I find the usual definitions far

too coarse and, as such, the resulting arguments in favor of the safety case
regime misleadingly convincing. 

Cheers,

Rene

From: systemsafety-bounces at lists.techfak.uni-bielefeld.de
[mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of
Les Chambers
Sent: woensdag 5 februari 2014 1:32
To: 'Peter Bernard Ladkin'; systemsafety at lists.techfak.uni-bielefeld.de
Subject: [SystemSafety] Safety Case Standards and Experience

Peter

Many thanks for this list. What a stout fellow you are. It's a great
reference or for anyone attempting to identify the current
"state-of-the-art" in safety cases.

I encourage anyone on the list who is aware of a safety case deliverable or
process standard, not identified here, to add to this thread.

Further, I encourage anyone with experience of preparing a safety case to
give us their thoughts.

Also, is anyone aware of system level product standards that may exist in
any application domain. Standards that mandate various high-level design
approaches as mentioned in Nancy's article ["Product: Specific design
features are required, which may be (a) specific designs or (b) more general
features such as fail-safe design or the use of protection systems."]. By
"system level" I mean pertaining to the overall design of the system, not a
component thereof. I would call an electrical installation a component. 

I am aware of texts such as "P. Clements et al., Documenting Software
Architectures: Views and Beyond, 2nd ed., Pearson Education, 2010" but have
not seen any ISO/IEC/CENELEC/DoD or other standards, in the public domain,
that could be called out in a development contract or used to certify
generic classes of systems. For these to be useful they need to be specific
enough to support a comply/not comply judgement. Because design approaches
are often tied to company intellectual property you don't often see this
stuff in the public domain. For example, in my chemical processing days, the
design of a latex reactor control system, at least at the strategic level,
was a copy and paste exercise. I could tell you about it but then I'd have
to kill you. 

In another life I spent a year leafing through the American Nuclear
Regulatory Commission standards with the objective of developing a NUREG
compliant system development methodology for a control system that would
perform emergency reactor shutdown. I did not encounter any constraints or
guidelines on the design approach. But that was some time ago.

I look forward to any and all contributions.

Les

---------------------------------------
>From Prof. Peter Bernard Ladkin, Faculty of Technology, University of
Bielefeld, 33594 Bielefeld, Germany

Oh, my!

A random Google search for the term "safety case" is as least as likely to
turn up this discussion
as anything else. That's not necessarily how you go about finding out what
the term might mean. (My
search turns up Nancy's paper in the first fifteen. Given the current
discussion, I am tempted to
regard that as *proof* that such a search isn't going to tell you what the
term might mean :-) ).

A *goal-oriented* (rather than random Google) search in the WWW for the
notion of "safety case"
could profitably start with the UK HSE, since Lord Cullen is the source both
for the original
recommendation concerning the notion of "Safety Case", in those words, and
the origin of the HSE.

One finds, on searching the HSE WWW site for the term "safety case",
references to the legislation
for offshore oil/gas installations, namely the England&Wales law known as
Offshore Installations
(Safety Case) Regulations 2005 (there are earlier versions). This law is
*very prescriptive* as to
what has to be included in an offshore installation Safety Case - you can
read it at
http://www.legislation.gov.uk/uksi/2005/3117/schedule/2/made

One also finds references to the legislation for railways, namely the
England&Wales law known as
Railways (Safety Case) Regulations 2003 (there are earlier versions). This
law is *prescriptive* as
to what has to be included in a railway Safety Case - you can read it at
http://www.legislation.gov.uk/uksi/2003/579/contents/made

It should be evident that a Safety Case for an offshore installation and a
Safety Case for a railway
are *two different things*, because these two enterprises use different kit:
pipes and pumps and
drills versus rails and locomotives and carriages. So rather than
> a term that has been defined for decades and is widely used to have a
specific meaning
there are at least two specific meanings in English law alone.

It is also absurd to suggest that either of these Safety Cases is just some
argument put together to
fulfil a goal-oriented safety standard. First, they are not there to fulfil
any standard, they are
the law of England and Wales (I say again: standards aren't laws). Second,
they are *prescriptive* -
what shall be in a Safety Case designated as such in these industries is
specified precisely in the
law.

So, sticking with Safety Cases with upper-case initial letters (or at least
with mid-size initials),
here is the definition of a Safety Case for the nuclear industry in Britain
(notice this is not a
law - the ONR surveys installations in Britain, and Britain has three
separate legal regimes:
England and Wales, Scotland, and Northern Ireland)

[begin quote from: Purpose and Scope of Safety Cases NS-TAST-GD-051 Revision
3]

5.    DEFINITION OF A NUCLEAR SAFETY CASE
5.1.    A nuclear safety case is a set of documents that describe the
radiological hazards in terms of
a facility or site and modes of operation (including potential undesired
modes) and the measures
that prevent or mitigate against harm being incurred. The safety case should
provide a coherent
demonstration that relevant standards have been met and that risks to
persons have been reduced to
as low as reasonably practicable (ALARP).

[end quote]

So there is, already, the third meaning. (To spell it out: neither of the
previous two regulations
"describe the radiological hazards" of a railway or an offshore
installation.) It is obviously not
as prescriptive as the first two, but it is still prescriptive.

So let's move to a generic definition, as embodied in a standard which is,
by the agreement forming
the European Union, the standard governing safety related electronic systems
for railway signalling
in 28 European countries (EN 50129:2003, the document referenced by Les
Chambers, in case readers
haven't pursued the link):

[begin quote EN 50129:2003 Section 3 Terms and Definitions]

3.1.49 safety case
the documented demonstration that the product complies with the specified
safety requirements

[end quote]

This is obviously a fourth definition, since it makes no reference to
pipes&pumps and stuff, or to
rails&locomotives and stuff, or to radiological hazards of a facility and
stuff. Unlike the first
three definitions, it is not prescriptive at all.

Then there is my definition, which might as well be this: a rigorous
argument, given in detail in a
specified set of documents which include all the evidence used in the
argument, that a system is
acceptably safe, both when it is operating and when it is not operating.

Notice that there is no requirement that all these documents have to reside
in one place. It covers
FAA and EASA certification practices and it is intended to cover the
capital-letter Safety Case
regimes above with a suitable interpretation of "acceptably safe".

I'll leave it as an exercise to point out the differences between my
definition and the previous four.

Here's another, from
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/27
585/SMP12v22final.pdf

[begin quote UK MoD SMP 12 Version 22, citing Def Stan 00-56 Version 4]

A structured argument, supported by a body of evidence that provides a
compelling, comprehensible
and valid case that a system is safe for a given application in a given
environment.

[end quote]

So, unlike offshore installations or UK railways, a given MoD system might
have a plethora of safety
cases, if there are a plethora of applications in a plethora of
environments. As well as depending
on the possibly-varying meaning of the system property "safe". (I'd guess
this is intended to mean
something similar to my term "acceptably safe".)

So much for "widely used to have a specific meaning"! More accurate would be
to use the plural. It
should also be evident that whether a safety case is narrowly prescriptive
or more free in form
depends on which industry, regulation and country you might be talking
about.

It's perfectly possible for there to be a good argument that the regime in
Offshore Installations
(Safety Case) Regulations 2003 in England & Wales "would not work" in the US
oil/gas industry, as
Nancy has indicated she believes. Such an argument would not obviously
generalise to, say, FAA
civilian aerospace certification documentation, which also forms a safety
case - indeed people like
me would say that that civilian aerospace safety cases are pretty good in
general, while pointing
out that they are by no means perfect, indeed it is possible to think that
overall they are pretty
good but specifically concerning avionics SW they are pretty poor.

It is also possible to consider FAA safety cases de re free-form (a
manufacturer can provide what
evidence it wants to establish a certification goal set by negotiation
between regulator and
manufacturer, for example on certification of lithium-ion batteries for the
Boeing 787) and de facto
prescriptive (if manufacturers have provided such-and-such documentation for
successful
certification of kit X in the past, then it is pretty much de rigeur that a
manufacturer will have
to provide at least the equivalent of that for future kit-X certification
proceedings. The
certification requirements, and therefore the recertification documentation,
after the Boeing 787
battery fires were *modified* from what they had been, not thrown away and
rewritten).

PBL

Prof. Peter Bernard Ladkin, Faculty of Technology, University of Bielefeld,
33594 Bielefeld, Germany
Tel+msg +49 (0)521 880 7319  www.rvs.uni-bielefeld.de

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20140217/930084bb/attachment-0001.html>