[SystemSafety] Additional Business Risk of Safety-Related Systems

[Peter Bernard Ladkin]

Folks,

I am working on some guidance material through the IET on safer complex systems. It came to a 
question as to what distinguishes the management layer (that is, activities) during development from 
management of a non-safety-related complex software system. I came to the following line of 
reasoning. Is it right? Helpful?

[begin reasoning]

The business risk to a developer of a complex software-based system consists inter alia in whether 
the system is built and delivered to the satisfaction of the client.
Such business risk is largely settled by formal acceptance of the system by the client, on the basis 
of acceptance testing. Of course, a supplier can (and usually does, if the system is bespoke and 
complex) continue to work on the system beyond acceptance, but this is often nominally classified as 
"maintenance" and enhancement. Once the system is accepted by the client, any business risk 
associated with development vanishes.

Business risk is assessed and controlled by the developer's internal audit, and guidance in best 
practice for internal audit is to be found on the WWW site of the Chartered Institute of Internal 
Auditors 
https://www.iia.org.uk/resources/delivering-internal-audit/position-paper-independence-and-objectivity/ 
. Inter alia it recommends that an Audit Committee also includes members of the company's board of 
directors.

However, building and supplying safety-related systems involves a business risk over and above this, 
in that, if a harmful event occurs anytime in system operation, some (or all) of the responsibility 
for that event may be deemed to lie with the system developer, regardless of the acceptance tests 
agreed and performed between customer and supplier. For example, if people die, the members of the 
board of the supplier may be criminally charged with corporate manslaughter. The possibility of such 
an event is an additional business risk.

This suggests that internal audit of safety-related system development must also ensure that 
appropriate practices have been followed to reduce the (technical) risk of system operation to be as 
low as reasonably practical (the ALARP legal condition in common law domains).

[Here we must note parenthetically that technical/system risk and business risk are two entirely 
different concepts. The IEC is currently grappling with all these different notions of risk. I 
identified 5 in common use in a talk I gave in September to the SCSC].

Since IEC 61508 has been available (1997 onwards), I understand the UK HSE has taken adherence to 
its precepts as the touchstone for whether (and how) to prosecute a supplier for the various forms 
of criminal negligence recognised in common law. So internal audit will also adjudge whether IEC 
61508 has been effectively followed during system development, and this activity may well proceed 
beyond initial system delivery, in particular in cases in which system use may be different from 
that initially envisaged and a need arises to ensure that this differential use is also adequately 
assured by the development.

[end reasoning]

PBL

Prof. i.R. Dr. Peter Bernard Ladkin, Bielefeld, Germany
Tel+msg +49 (0)521 880 7319  www.rvs-bi.de




-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://lists.techfak.uni-bielefeld.de/pipermail/systemsafety/attachments/20220425/3a4ef8fa/attachment.sig>