[SystemSafety] Component Reliability and System Safety

Peter Bernard Ladkin ladkin at causalis.com
Tue Sep 18 08:43:45 CEST 2018



On 2018-09-17 17:29 , Paul Sherwood wrote:
>
> On 2018-09-17 13:19, Peter Bernard Ladkin wrote:
>> On 2018-09-17 11:06 , Paul Sherwood wrote:
>>> But software is a very big field. It seems to me that most of the software we are relying on these
>>> days was developed without following coding standards in general, ....
>>
>> That may be true in general; I wouldn't know. It is specifically not
>> true for safety-related systems.
> 
> Hmmm. I'm sure you mean something specific by "safety-related systems" but I think my understanding
> must be different. Is there a succinct and agreed definition anywhere?

Yes. It is a key term in the functional safety standard for E/E/PE systems, IEC 61508.

[begin quote]

3.4.1
 safety-related system

designated system that both
 –– implements the required safety functions necessary to achieve or maintain a safe state for the
EUC; and
 –– is intended to achieve, on its own or with other E/E/PE safety-related systems and other risk
reduction measures, the necessary safety integrity for the required safety functions

NOTE 1 The term refers to those systems, designated as safety-related systems, that are intended to
achieve, together with the other risk reduction measures (see 3.4.2), the necessary risk reduction
in order to meet the required tolerable risk (see 3.1.7). See also Annex A of IEC 61508-5.

NOTE 2 Safety-related systems are designed to prevent the EUC from going into a dangerous state by
taking appropriate action on detection of a condition which may lead to a hazardous event. The
failure of a safety-related system would be included in the events leading to the determined hazard
or hazards. Although there may be other systems having safety functions, it is the safety-related
systems that have been designated to achieve, in their own right, the required tolerable risk.
Safety-related systems can broadly be divided into safety-related control systems and safety-related
protection systems.

NOTE 3 Safety-related systems may be an integral part of the EUC control system or may interface
with the EUC by sensors and/or actuators. That is, the required safety integrity level may be
achieved by implementing the safety functions in the EUC control system (and possibly by additional
separate and independent systems as well) or the safety functions may be implemented by separate and
independent systems dedicated to safety.

NOTE 4 A safety-related system may
 a) be designed to prevent the hazardous event (i.e. if the safety-related systems perform their
safety functions then no harmful event arises);
 b) be designed to mitigate the effects of the harmful event, thereby reducing the risk by reducing
the consequences;
 c) be designed to achieve a combination of a) and b).

NOTE 5 A person can be part of a safety-related system. For example, a person could receive
information from a programmable electronic device and perform a safety action based on this
information, or perform a safety action through a programmable electronic device.

NOTE 6 A safety-related system includes all the hardware, software and supporting services (for
example, power supplies) necessary to carry out the specified safety function (sensors, other input
devices, final elements (actuators) and other output devices are therefore included in the
safety-related system).

 NOTE 7 A safety-related system may be based on a wide range of technologies including electrical,
electronic, programmable electronic, hydraulic and pneumatic.

[end quote]

> Are any IoT devices "safety-related systems" in your understanding? Is a car a safety-related
> system? For both of those classes of device, the majority of code in them is not MISRA C.

My understanding is not that relevant here. Whether anything constitutes a safety-related system
depends on whether it fulfils the two clauses above. Both of your questions are too general to be
canonically answered according to the definition.

>> IEC 61508-3 Table B.1 entry 1 says that "Use of coding standard to
>> reduce errors" is Highly
>> Recommended (HR) for all SILs. ("Highly Recommended" is the strongest
>> form of encouragement for any
>> specific technology in the standard).
> 
> Hmmm, again.
> 
> I confess I haven't read 61508 myself (and do not expect to, given its cost and EULA) but I'm
> currently hopeful that I can find useful and practical guidance elsewhere, including via this list.

If you are going to be working on systems which have a functional safety aspect in the UK, the
regulator HSE will require that you are familiar with and conformant with IEC 61508 (as it has done
for two decades). Exceptions are for military use (the MoD has its own standards) and those civil
industries which have a domain-specific standard, such as rail (EN 5012x) , road vehicles (ISO
26262), medical devices (yyyyy) and civil aviation (ED-12C/ED-217/ED-218 for avionics, EC 482-2008,
ED-109 for ground-based systems).

There is lots that is not quite right with most of these standards. But they do rule the roost. It
therefore seems worthwhile to work to improve them. Sometimes, though, things go backwards. There
seems to be some chance that cybersecurity will be badly fumbled in the next edition of IEC 61508.

>> If you don't use a coding standard, your assessor is going to want to
>> know why. (Telling himher you
>> think they are outmoded is not generally regarded as an acceptable answer.)
> 
> I'm more surprised that "we follow a coding standard" would be considered as an acceptable answer to
> anything that impacts on system safety.
> 
> I wouldn't accept it myself, frankly, since I've been told lies by many people defending their
> projects over the years.

I think you are missing my point. Someone looking at your safety-related software in the UK is going
to be assessing it against IEC 61508-3. IEC 61508-3 Annex B Table B.1 says that following a coding
standard is highly recommended for implementation of any safety function in software. If you didn't
do it, a conscientious assessor is going to be asking why not. I doubt that your above two comments
would be seen to constitute a reasonable response.

>>> We could insist that the software be developed in Haskell, or Rust, or some other technology that
>>> provides a higher level of control over the code creation.
>>
>> Not in developing safety-related systems, you can't. At least, not at
>> the moment. With Haskell, the
>> need for garbage collection gets in the way of being used (it
>> potentially interferes with timing
>> constraints in a non-deterministic way).
> 
> Hmmm. This seems to be another throw-back from decades ago. I expect it's possible to design for
> safety without requiring that all applicable software satisfies time constraints in a deterministic
> way?

Gee, where to begin? Let's try this. Unstable aircraft-pilot coupling phenomena can manifest
themselves through to loss of control of an aircraft in small fractions of a second. It is also not
currently possible to identify all possibilities for unstable APC in a control system design
statically. You can't possibly have your FCC garbage-collecting when it is supposed to be reacting
to aircraft sensor data at a 200Hz frequency.

>> Since the general E/E/PE functional safety standard HR's use of coding
>> standards, what could be the
>> scope of "used when they shouldn't be"?
> 
> Well, the simple in-context example here would be a case where folks misguidedly insist that
> something should be created in MISRA C, without considering potential alternative approaches that
> didn't require it, e.g.:

You surely mean "potential alternative coding standards", since using a coding standard is HR.

>>> dependable != safe
>>
>> Thank you.
> 
> I take it you agree, and knew this already, and were being ironic.

These are technical terms with various definitions in the literature. The most used definitions are
IEC 60050 (the IEV, on-line at http://www.electropedia.org), where the definition of dependability
is 192-01-22 (Section 192 is Dependability) and IFIP WG 10.4, whose definitions are in Laprie et al.
(eds.) Dependability: Basic Concepts and Terminology, Springer-Verlag 1991, and a later article
Avizienis et al., Basic Concepts and Taxonomy of Dependable and Secure Computing, IEEE Trans. Dep.
Sec. Comp. 1(1).

BTW, according to the IEC, dependability does not include safety. See
http://www.iec.ch/dyn/www/f?p=103:7:0::::FSP_ORG_ID:1270   According to IFIP, it does.

There are also some definitions in Leveson, Safeware, Addison-Wesley 1995, Chapter 9. And I presume
in "Engineering...Safer".


>> Current best practice involves following the strictures of IEC 61508
>> or its so-called "derivatives"
>> for non-aerospace, non-medical systems, and ISO 26262 for road
>> vehicles; DO-178C/EA-12C for critical
>> aerospace code, including EA-217 and EA-218. (I have forgotten the
>> numbers of the medical-systems
>> E/E/PE safety standards. They come largely from IEC TC 62 and 66, I think.)
> 
> That being the case, and assuming that the IEC document is actually useful (I have some concerns
> about that, as I am sure you already understand) why is IEC 61508 so ridiculously expensive that
> most engineers will never read it?

That is an issue which John Knight, I, Martyn Thomas, and others have discussed in public for a long
time.

It is the explicit business model of most standards organisations in Western countries, including
the ISO and IEC. The trivial answer is that IEC 61508 is so expensive because that is what the
costing model says it has to cost in order to support the IEC and all its staff and buildings. The
less trivial answer is that the business model is flawed in a variety of respects.

I don't see any easy way to change this.

> Apart from the various ecosystem players making money from the overall scheme I do think that in
> part the underlying aim has been to maintain credibility via mystique and avoiding public scrutiny.
> Given you're maintaining it I expect you may be unhappy with that comment, but the fact remains - if
> the document is really what people should be reading, then the 3K tax per reader is a disgrace.

I agree with the principle of what you are saying here. I have advocated that each FDIS should be
subject to expert peer review before being published. The EU does this with each financed project,
and I have experienced it to be a very effective form of quality control, while being inevitably
imperfect. There is a fair amount of junk engineering in large standards, and such peer review could
get rid of a lot of it. For my take on some of the junk, see
https://rvs-bi.de/publications/RVS-Bk-17-01.html There is a fair amount of self-interested pressure
applied by large companies in standards processes to get their view enshrined in a standard.
Independent peer review would temper a lot of that. (I think large industry players know this well,
and would thus be against such review.)

But I do not agree that the "aim has been to maintain credibility via mystique and avoiding public
scrutiny". Nobody I know on the committees gives a fig about "mystique". It is just a committee like
any other. And the issue with public scrutiny is that we want as much as we can get, and so does the
IEC as far as I can tell. There are indeed opportunities for public comment on drafts by any
engineer. But National Committees have their own politics and ways of restricting commentary. Few
engineers get to know that a CD has been published and comments from anyone may be given in the
three-month comment period. The reason for that is that National Committees/National Standards
Organisations do not make it known. Neither do they necessarily make CDs easily available for
engineers to comment. And and such comments are vetted by the NCs before being passed to the IEC.
There is lots of national politics going on in this process, and some of it is miserable.

>>> If our safety depends on the reliable behaviour of even a small program on (say) a modern multi-core
>>> microprocessor interacting with other pieces of software in other devices, I think "we are lost"
>>> again.
>>
>> It does, and it will do for the foreseeable future.
> 
> Did you miss my point about "modern multi-core microprocessor"? 

No, I didn't. There is a task group on the Maintenance Teams addressing it. I think it has two members.


>> You may also not know what safety-related systems look like. Lots of
>> little and mid-sized boxes
>> plugged together would not be an inappropriate picture.
> 
> OK so from this you're not talking about (say) a pacemaker.

Nope. That is a medical device, and they have their own regulation. General stuff is by and large
not applicable. I think there are five makers of pacemakers. The engineering around pacemakers is
pretty sophisticated.

> <snip>
>> A colleague observed
>> that many or most of the CERT
>> advisories still concern phenomena that could have been avoided
>> through the use of (rigorous,
>> reliable) strong typing.
> 
> Fine - we can enforce via tooling and/or better languages, rather than a coding standard.

>>> In automotive I know that some user-facing (and even internet-facing) systems *do* sit on the CAN
>>> bus, .....
>>
>> *The* CAN bus?
> 
> Sorry, I'm not sure what you mean by this comment. 

Road vehicles based on CAN have large bundles of such buses.

PBL

Prof. Peter Bernard Ladkin, Bielefeld, Germany
MoreInCommon
Je suis Charlie
Tel+msg +49 (0)521 880 7319  www.rvs-bi.de





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180918/f4319fba/attachment.sig>


More information about the systemsafety mailing list