[SystemSafety] The Rush

[SPRIGGS, John J]

Warning a regulator that he might get caught with a jumbuck in his tucker-bag will certainly give him pause for thought...

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Les Chambers
Sent: 09 December 2016 01:18
To: systemsafety at lists.techfak.uni-bielefeld.de
Subject: [SystemSafety] The Rush

Hi
There are many things that can spook a herd of bullocks. A possum, a cracking twig, a flash of lightning or a bush rat chewing on a tender hoof. One minute the herd is at peace under the stars and the next the're up and rushing. The Americans call this a stampede, in Australia it's a rush. All drovers have seen this many times and they know how to shut one down. They have to before it rolls over the camp site and kills sleeping men in their swags. Money is at stake too. You can lose cattle in a bad rush and a drover gets paid on how many bullocks he delivers at the other end , sometimes more than 2000 kilometres down the track.
When a rush happens a drover is often alone. One man circling a camped herd of 1500 on a dark night watching for strays while his mates get some sleep. Here's the thing. A rushing herd always moves in one direction with the fast movers (thought leaders) out front and the slower followers bringing up the rear. The trick is to get to the leaders and turn them in a circle back on the herd so you end up with a heaving mass of sirloin going round and round and round. Even the most primitive beasts can see the futility of this so they ultimately calm down and the rush is over. It sounds simple but it's not. You'd better be a skilled stockman and, if it's night, be on a good 'night horse', one with good night vision and surefootedness in the dark. These horses are prized among drovers, they save lives.

I heard all this from an old drover interviewed on Australia's ABC radio national and I couldn't resist the metaphor. It so accurately describes the state of technology today and hints at what we should do.

Technology today is in a rush. Spooked by the odd possum (read: imagined pot of gold) various risktakers are up and bolting to nowhere in particular tearing through the scrub more focused on the act of galloping (read: I'm doing it because I can) rather than the destination or the consequences. And in the process much of what we've learnt in the past 50 years about making systems safe and secure is being ignored either through ignorance or greed or man's unquenchable desire to slaughter his enemies, be they logical or physical. This became clear at the IET's London conference on safety and security this October. Where I was shocked to learn of the very basic security measures that ARE NOT being taken by many organisations deploying web apps in critical applications such as electricity infrastructure. For example when I asked if SCADA products with Internet connectivity are delivered standard with firewalls the presenters laughed. Ha, ha. Then we spent an hour listening to the disembodied voice (via Skype) of a Ukrainian cyber warrior detailing how Russian hackers shut down a Ukrainian power network turning out the lights on 250,000 people. It seems the Internet has been a war zone for some time now.

So where are the stockman on their night horses at the head of the rush turning the wrong thinking thought leaders back into the pack where we hope rationalism will prevail. At the conference, UK government representatives talked of 'kinetics'. "We will no longer just defend. We plan to attack the source of hacks," they said. Naming and shaming was also popular. Then there are the regulators. I asked one of them what they plan to do about Tesla's fraudulent proven-in-use claims of reliability and safety on a vehicle that has regular software upgrades. He referred me to a website. "Fill out the form," he said. He reminds me of a drover who climbs a tree when a rush is on.

So what should we do? My thought is that all professionals have their surefooted night horses. The courses of action that we know work (and the ones that we know will certainly end in tears). All we have to do is trust in them just as a galloping stockman trusts the animal beneath him in the pitch black amongst a herd of panicked 1000 kg bullocks (picture Elon Musk at full stretch). And with that certainty we head for the leaders of the rush and aggressively turn them. That is, aggressively attack bad ideas where ever they occur.

For example take the statement:
"Suppose you have some software whose performance you have assessed through operational experience and testing. You know to 95% confidence that the software has a failure likelihood of less than 1 in 10^(-6) per operational hour."

My night horse, the one I've been riding for 40 years, tells me that this statement has a subtext, a way of thinking, that needs to be turned: the notion that you can derive a numerical indicator of failure likelihood on software as a function of experience and testing.
This is my belief because:
1. Every single body of software I have dealt with for the last 40 years has, throughout its operational life, been constantly subject to upgrade. Changing one line of code invalidates any previous confidence gained from operational experience (this seems to be lost on Tesla's marketing department). My worst scare was a one LOC defect that nearly destroyed a chemical reactor.
2. Testing does not confirm the absence of all defects.
3. Regression testing, post-maintenance, is universally poorly done. There will always be a conflict of interest between quality and money.
4. Given that most complex systems these days run into millions of SLOCs and use a variety of software libraries over which the developer has no control, paranoia is a better state of mind than confidence.

Depending on the above notion to claim software safety is like giving a bulloch an IQ test to claim it can make it to the rail head 2000 kilometres away without a drover. Which brings me to my point. If confidence can be attained at all it is by examining the drovers who manage the herd together with their excellent night horses. Back in the day this was my understanding of the thrust of 61508 - "... if the processes are best practice and the people are highly skilled we will allow you to use the software in critical applications. ..." I hope this has not changed.

And as for the rest of ya, wake up, get out of your swags, the cognitive rush is on in a new silly season of software development, I'm nearing the leader, bolting because he can, admiring the mathematical brilliance of his predictions on the behaviour of the 10 line program when the rest of us cattlemen have to find our way in the dark to deal with a herd of millions. I'm turning, I'm turning him. Help me, you will be rewarded as Banjo Paterson said:

And the bush hath friends to meet him, and their kindly voices greet him
In the murmur of the breezes and the river on its bars,
And he sees the vision splendid of the sunlit plains extended,
And at night the wond'rous glory of the everlasting stars.
(From Clancy of the Overflow)

Merry Christmas all

Les

-------------------------------------------------
Les Chambers
Director
Chambers & Associates Pty Ltd
www.chambers.com.au<http://www.chambers.com.au>
Blog: www.systemsengineeringblog.com<http://www.systemsengineeringblog.com/>
Twitter: @ChambersLes<http://www.twitter.com/chambersles>
M: 0412 648 992
Intl M: +61 412 648 992
Ph: +61 7 3870 4199
Fax: +61 7 3870 4220
les at chambers.com.au<mailto:les at chambers.com.au>
-------------------------------------------------

***************************************************************************
If you are not the intended recipient, please notify our Help Desk at Email information.solutions at nats.co.uk
immediately. You should not copy or use this email or attachment(s) for any purpose nor disclose
their contents to any other person.

NATS computer systems may be monitored and communications carried on them recorded, to 
secure the effective operation of the system.

Please note that neither NATS nor the sender accepts any responsibility for viruses or any losses
caused as a result of viruses and it is your responsibility to scan or otherwise check this email
and any attachments.

NATS means NATS (En Route) plc (company number: 4129273), NATS (Services) Ltd 
(company number 4129270), NATSNAV Ltd (company number: 4164590) 
or NATS Ltd (company number 3155567) or NATS Holdings Ltd (company number 4138218). 
All companies are registered in England and their registered office is at 4000 Parkway, 
Whiteley, Fareham, Hampshire, PO15 7FL.

***************************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20161209/e4858fb0/attachment.html>