[SystemSafety] The Rush - we and the dead

[Les Chambers]

In my experience the main issue that rules out agile in safety critical work is the overhead of completing safety critical life cycle activities for multiple release strategies. If you can get through them once every 2 to 3 weeks, more power to you. But without massive automation in the development shop this is just not realistic. The alternative, the one that scares me, is taking shortcuts and not bothering to much about all that stuff.
The other constraint is the nature of most safety critical applications. They are often large and cannot be delivered in small pieces. And they are often hard to get at or impossible to get at to perform the upgrade. Imagine the hellfire missile developer shouting , "hey bring back that missile I've got some new features that the enemy will love."
We've just seen an example of this with the Internet of things. In the stampede to get product out the door we are now stuck with thousands of insecure bots that can be easily enslaved to perform denial of service attacks.

The main thrust of safety critical work and the point of the plethora of standards we have is to build systems that are correct by design. To think about the ramifications of what you're doing before you do it. To think about architecture, emergence and the possibility of unintended consequences. This seems to be at odds with the "ready fire aim" agile mentality. I know this is incredibly uncool and so 20th-century but it is fundamentally true. 

This year my Christmas reading  includes a book called The Wayfinders , Wade Davis. It's log line reads "why ancient wisdom matters in the modern world". It turns out that for centuries the tribes of the Amazon River Valley have felt the need for both a priest and a shaman. The shaman deals laterally with the forces of nature which are in constant flux and the priest deals vertically with the gods and the ancestors. The priest deals in fundamental truths that do not change, the shaman is a mercurial character focused on overcoming immediate and always changing short-term problems. Usually solving them with short-term measures, charms, chants, spells and the like. He is typically a charismatic who holds sway over the weak minded , the easily influenced. 

To all on this list: Which one are you?


> On 22 Dec. 2016, at 8:39 pm, Mike Ellims <michael.ellims at tesco.net> wrote:
> 
> Having looked at this I suspect that the article is mostly spin and in some parts quite misleading e.g.  Exhibit 3, what do they mean by Tech Players, all companies that can be classed as technology companies e.g. Oracle, Cisco, Microsoft? Only a small proportion of these have anything to do with automotive e.g. Uber which so far doesn’t appear to have actually made a profit.
>  
> It also ignores the fact that some of the tech start-ups have been brought out by the likes of GM (one example).
>  
> Quite a lot of the “article” is just so much hand waving, for example:
>  
> “For example, automakers reengineer their core products approximately once every seven years, with noticeable updates every three years, but do not update existing products.”
> Actually, updates while not common are not unknown and are usually applied during routine servicing with releases every 6-12 months if a recall is not required. This is normally limited to tweaks to existing functionality rather than new functionality. For example the first editions of the Mk II  Ford Mondeo (I have one of these) had an issue with stalling on cold start which was fixed by small changes to the calibration.
>  
> “The OEMs’ systematic “waterfall” approach to product development tends to slow down innovation;”
> I’ve done work for several OEM’s in my time. Not one used a waterfall model. The dominant framework was the V model for new developments with again about six months between major releases up the ready for production mark. So some aspects of this already approximate an agile methodology.
>  
> Oh and merry Xmas :-)
>  
> From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Les Chambers
> Sent: 22 December 2016 00:36
> To: 'SPRIGGS, John J'; systemsafety at lists.techfak.uni-bielefeld.de
> Subject: Re: [SystemSafety] The Rush - we and the dead
>  
> More on the Rush
> This McKinsey article on ecosystems in the new smart/agile/innovative/intelligent/always-on (everything, all the time) auto industry is symptomatic of the rush I mentioned in a previous post. The auto industry is stampeding into tech, spurred on by various consultancies, staffed with children from Silicon Valley. I have included some scary excerpts below. My concern is that in damning rigid/rigourous/waterfall/non-agile product development processes they are, in fact, talking about us. The "cruel dudes" who will stop a production release if it has not gone through a proper (rigid/non-agile/rigourous/professional/exacting/detail driven/risk-driven/waterfall) safety life cycle.
>  
> Read this and be very afraid:
> http://www.mckinsey.com/industries/automotive-and-assembly/our-insights/how-the-convergence-of-automotive-and-tech-will-create-a-new-ecosystem?cid=other-eml-alt-mip-mck-oth-1611
>  
> Some classic wisdom from Gottfried August Burger is relevant here:
>  
> Look abroad – the moon shines bright,
> We and the dead ride fast by night
>                                                           Lenore
>  
> ------ classic quotes from the child consultants  ----
> They often adhere to rigid, rigorous, and unique product-development practices; work with complex supply chains; and sell through extensive franchised retail-dealer networks.
>  
> Tech players prefer experimental, fast-moving cultures that reward innovation and risk taking.
>  
> Tech companies redo their core products about every two years, make noticeable updates every two months, and provide continual updates for existing products.
>  
> The OEMs’ systematic “waterfall” approach to product development tends to slow down innovation; the average time to market is about five years. Most tech players depend on agile operating models that enable a time to market of roughly two years.
>  
> The convergence of the automotive and high-tech sectors will rewrite the rules of competition and lessen the chances of survival for traditional players that fail to act.
>  
> ------ end of classic quotes --------
>  
> From: SPRIGGS, John J [mailto:John.SPRIGGS at nats.co.uk] 
> Sent: Friday, December 9, 2016 11:52 PM
> To: 'Les Chambers'; 'systemsafety at lists.techfak.uni-bielefeld.de'
> Subject: RE: [SystemSafety] The Rush
>  
> Warning a regulator that he might get caught with a jumbuck in his tucker-bag will certainly give him pause for thought…
>  
> From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Les Chambers
> Sent: 09 December 2016 01:18
> To: systemsafety at lists.techfak.uni-bielefeld.de
> Subject: [SystemSafety] The Rush
>  
> Hi
> There are many things that can spook a herd of bullocks. A possum, a cracking twig, a flash of lightning or a bush rat chewing on a tender hoof. One minute the herd is at peace under the stars and the next the're up and rushing. The Americans call this a stampede, in Australia it's a rush. All drovers have seen this many times and they know how to shut one down. They have to before it rolls over the camp site and kills sleeping men in their swags. Money is at stake too. You can lose cattle in a bad rush and a drover gets paid on how many bullocks he delivers at the other end , sometimes more than 2000 kilometres down the track.
> When a rush happens a drover is often alone. One man circling a camped herd of 1500 on a dark night watching for strays while his mates get some sleep. Here's the thing. A rushing herd always moves in one direction with the fast movers (thought leaders) out front and the slower followers bringing up the rear. The trick is to get to the leaders and turn them in a circle back on the herd so you end up with a heaving mass of sirloin going round and round and round. Even the most primitive beasts can see the futility of this so they ultimately calm down and the rush is over. It sounds simple but it's not. You'd better be a skilled stockman and, if it's night, be on a good 'night horse', one with good night vision and surefootedness in the dark. These horses are prized among drovers, they save lives.
>  
> I heard all this from an old drover interviewed on Australia's ABC radio national and I couldn't resist the metaphor. It so accurately describes the state of technology today and hints at what we should do.
>  
> Technology today is in a rush. Spooked by the odd possum (read: imagined pot of gold) various risktakers are up and bolting to nowhere in particular tearing through the scrub more focused on the act of galloping (read: I'm doing it because I can) rather than the destination or the consequences. And in the process much of what we've learnt in the past 50 years about making systems safe and secure is being ignored either through ignorance or greed or man's unquenchable desire to slaughter his enemies, be they logical or physical. This became clear at the IET's London conference on safety and security this October. Where I was shocked to learn of the very basic security measures that ARE NOT being taken by many organisations deploying web apps in critical applications such as electricity infrastructure. For example when I asked if SCADA products with Internet connectivity are delivered standard with firewalls the presenters laughed. Ha, ha. Then we spent an hour listening to the disembodied voice (via Skype) of a Ukrainian cyber warrior detailing how Russian hackers shut down a Ukrainian power network turning out the lights on 250,000 people. It seems the Internet has been a war zone for some time now.
>  
> So where are the stockman on their night horses at the head of the rush turning the wrong thinking thought leaders back into the pack where we hope rationalism will prevail. At the conference, UK government representatives talked of 'kinetics'. "We will no longer just defend. We plan to attack the source of hacks," they said. Naming and shaming was also popular. Then there are the regulators. I asked one of them what they plan to do about Tesla's fraudulent proven-in-use claims of reliability and safety on a vehicle that has regular software upgrades. He referred me to a website. "Fill out the form," he said. He reminds me of a drover who climbs a tree when a rush is on.
>  
> So what should we do? My thought is that all professionals have their surefooted night horses. The courses of action that we know work (and the ones that we know will certainly end in tears). All we have to do is trust in them just as a galloping stockman trusts the animal beneath him in the pitch black amongst a herd of panicked 1000 kg bullocks (picture Elon Musk at full stretch). And with that certainty we head for the leaders of the rush and aggressively turn them. That is, aggressively attack bad ideas where ever they occur.
>  
> For example take the statement:
> "Suppose you have some software whose performance you have assessed through operational experience and testing. You know to 95% confidence that the software has a failure likelihood of less than 1 in 10^(-6) per operational hour."
>  
> My night horse, the one I've been riding for 40 years, tells me that this statement has a subtext, a way of thinking, that needs to be turned: the notion that you can derive a numerical indicator of failure likelihood on software as a function of experience and testing.
> This is my belief because:
> 1. Every single body of software I have dealt with for the last 40 years has, throughout its operational life, been constantly subject to upgrade. Changing one line of code invalidates any previous confidence gained from operational experience (this seems to be lost on Tesla's marketing department). My worst scare was a one LOC defect that nearly destroyed a chemical reactor.
> 2. Testing does not confirm the absence of all defects.
> 3. Regression testing, post-maintenance, is universally poorly done. There will always be a conflict of interest between quality and money.
> 4. Given that most complex systems these days run into millions of SLOCs and use a variety of software libraries over which the developer has no control, paranoia is a better state of mind than confidence.
>  
> Depending on the above notion to claim software safety is like giving a bulloch an IQ test to claim it can make it to the rail head 2000 kilometres away without a drover. Which brings me to my point. If confidence can be attained at all it is by examining the drovers who manage the herd together with their excellent night horses. Back in the day this was my understanding of the thrust of 61508 - "... if the processes are best practice and the people are highly skilled we will allow you to use the software in critical applications. ..." I hope this has not changed.
>  
> And as for the rest of ya, wake up, get out of your swags, the cognitive rush is on in a new silly season of software development, I'm nearing the leader, bolting because he can, admiring the mathematical brilliance of his predictions on the behaviour of the 10 line program when the rest of us cattlemen have to find our way in the dark to deal with a herd of millions. I'm turning, I'm turning him. Help me, you will be rewarded as Banjo Paterson said:
>  
> And the bush hath friends to meet him, and their kindly voices greet him
> In the murmur of the breezes and the river on its bars,
> And he sees the vision splendid of the sunlit plains extended,
> And at night the wond'rous glory of the everlasting stars.
> (From Clancy of the Overflow)
>  
> Merry Christmas all
>  
> Les
>  
> -------------------------------------------------
> Les Chambers
> Director
> Chambers & Associates Pty Ltd
> www.chambers.com.au
> Blog: www.systemsengineeringblog.com
> Twitter: @ChambersLes
> M: 0412 648 992
> Intl M: +61 412 648 992
> Ph: +61 7 3870 4199
> Fax: +61 7 3870 4220
> les at chambers.com.au
> -------------------------------------------------
>  
>  
> 
> If you are not the intended recipient, please notify our Help Desk at Email Information.Solutions at nats.co.uk immediately. You should not copy or use this email or attachment(s) for any purpose nor disclose their contents to any other person. 
> 
> NATS computer systems may be monitored and communications carried on them recorded, to secure the effective operation of the system. 
> 
> Please note that neither NATS nor the sender accepts any responsibility for viruses or any losses caused as a result of viruses and it is your responsibility to scan or otherwise check this email and any attachments. 
> 
> NATS means NATS (En Route) plc (company number: 4129273), NATS (Services) Ltd (company number 4129270), NATSNAV Ltd (company number: 4164590) or NATS Ltd (company number 3155567) or NATS Holdings Ltd (company number 4138218). All companies are registered in England and their registered office is at 4000 Parkway, Whiteley, Fareham, Hampshire, PO15 7FL.
> 
> 
>   			 			
> This email has been checked for viruses by Avast antivirus software. 				
> www.avast.com
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20161223/02d7f061/attachment-0001.html>