[SystemSafety] Another unbelievable failure (file system overflow)

Martyn Thomas martyn at thomas-associates.co.uk
Mon Jun 1 10:34:53 CEST 2015


Les/Steve

Thanks for this. There's little discussion of professional ethics in any
forum that I read.

Do you think there's any hope that we might be able to make a small
advance in a focused area, such as IEC 61508? The standard isn't fit for
purpose, in that it largely ignores cybersecurity issues and does not
provide a sound basis for assessing whether safety-critical systems are
safe enough for their proposed application. It's also too long,
inconsistent, too expensive, and can't be copied/republished for use in
teaching, research or professional debate. I see these weaknesses, in
the central international standard for the safety of computer-based
systems, as an ethical issue. Do you agree?

Regards

Martyn

On 31/05/2015 05:14, Les Chambers wrote:
>
> Steve
>
> Thanks for referencing the code of ethics. It should be brought up
> more often. Unfortunately, for me, it makes depressing reading.
> Especially when you come upon paragraphs such as:
>
>  
>
> 3.12. Work to develop software and related documents that respect the
> privacy of those who will be affected by that software.
>
>  
>
> Although he has probably never read it, there is a man, who will
> probably never see his homeland again because he took these sentiments
> to heart and attempted his own corrective action. And what of the
> thousands of scientists, engineers and technologists who contributed
> to the construction of the software, the existence of which, he
> exposed to the world?
>
>  
>
> My point is that non-compliance with this code of ethics is massive
> and almost universal. In fact, any engineer maintaining strict
> compliance with every paragraph of this code would be unemployable in
> our modern world.
>
>  
>
> Reading these paragraphs through the lens of experience I am blown
> away by their flippancy. From personal experience I can tell you that
> screwing up the courage to implement even one of these items can be a
> massive life changing event. This sentence would be lost on a
> graduate. They're all perfectly reasonable statements of how one
> should behave. Much like, "Thou shall not kill, thou shall not commit
> adultery ...".  The issue lies in the moral courage to implement.
>
>  
>
> There is no quick fix to this problem as we are a decentralised,
> unorganised and generally fragmented lot. We don't have the luxury of
> the medical profession that deals with a single organism. We can't
> simply state and righteously comply with the notion of, "Do no harm."
> In fact, for us, the opposite is true, many of us work in industries
> where the primary purpose is to kill other human beings, and with high
> efficiency (fewer soldiers kill more enemy).
>
>  
>
> One thing we can do is deal with the problem at its root:
>
>  
>
> We are graduating incomplete human beings from science and engineering
> courses. There is insufficient focus on the moral issues surrounding
> the impact of our machines on humanity. For example, a study of
> applied philosophy, including ethics, should be a nonnegotiable
> component of all engineering courses. Not just a final year subject,
> but a subject for every year with a weekly reflection on the content.
> Much like the weekly safety meetings I was forced to attend in the
> chemical processing industry.
>
>  
>
> I'm sure there will be howls of laughter at this, but, let me tell you
> it's the only thing that caused me to back a senior manager about five
> levels above my pay grade into a corner - he could physically not
> escape me short of punching me out and stepping over my body - and
> berate him until he promised to properly train his operators in the
> emergency procedures for a safety critical system.
>
>  
>
> Popping a few paragraphs up on the web would never have done the trick.
>
>  
>
> That experience was trivia compared to where we are headed. The
> massive computing power now available means that our software is
> beginning to take higher level decisions away from human beings. Some
> of these decisions are moral ones (refer my previous post on lethal
> autonomous weapons systems). "Shall I kill all humans associated with
> this structure, or no?"
>
>  
>
> At a recent engineering alumni meeting I asked the head of my old
> engineering Department how much philosophy is taught to undergraduate
> engineers. He chuckled. "It is available as an elective but less than
> one percent participate," he said.
>
>  
>
> I plan to speak to him again soon.
>
>  
>
> Cheers
>
> Les
>
>  
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20150601/95b3de53/attachment.html>


More information about the systemsafety mailing list