[SystemSafety] Hackers take over control of a car wirelessly

Thu Jul 23 13:14:06 CEST 2015

Apologies Les (everyone),  this is quite long :-(

> 1. All Musk companies operated (and may still be operating) under the constant threat of bankruptcy (situation normal for a start-up).

> The temptation to take shortcuts would have been huge (at least in the early days – and possibly still today).

Musk has been at this for a while, he’s a serial investor e.g. PayPal which seems to have been moderately successful if unbelievably annoying. There is also evidence to the contrary i.e. they have delayed the introduction of the Model X I think 3 times now because it wasn’t ready,  so it’s something like eighteen months late.

There has also been some “de-risking” with the Model X  in that the drive train for the vehicle has been fitted to the 4x4 S Type, they seem to have a plan there somewhere.

> 2. Eighty to one hundred hour work weeks are normal. New hires are warned that, "this is special forces territory". I tried this once for

> a year and discovered that you make dangerous design decisions late at night. After a year of this I turned into a gibbering idiot and took

> six months to recover. This leads me to my next point:

> 3. Musk burns out his people. His turnover is high. The only people with the stamina to sustain this workload are probably 20 and 30

>  something’s who are learning on-the-job. You wonder what kind of design decisions they are making at midnight.

See below...

>  4. Disagreeing with musk is career limiting, proving him wrong is fatal. He is not what you'd call a well adjusted kind of guy.

If he was a normal well adjusted kind of guy a) he wouldn’t be a self made billionaire b) wouldn’t have touched an electric car start up with a bargepole [The automotive industry truism is that if you want to LOSE a billion dollars start a car company] and c) we wouldn’t be discussing Tesla now and they wouldn’t be getting headlines like this... 

http://www.albertaoilmagazine.com/2015/07/is-teslas-model-s-the-beginning-of-the-end-for-oil/

As George Bernard Shaw said; “Reasonable people adapt themselves to the world. Unreasonable people attempt to adapt the world to themselves. All progress, therefore, depends on unreasonable people.”

I strongly suspect that Musk is on the high end of unreasonable. It doesn’t make him either stupid or incompetent, though it doesn’t rule out crazy.

I imagine that Musk and Co. know they are the target of every other major automotive manufacture and the whole of the fossil fuel industry and consequently a) need to get where they are going as quickly as possible b) that they can’t screw up because that’s the end of them and quite possibly the electric vehicle industry.

> Try  establishing and running a consistent safety engineering regime in this kind of environment. Musk reacts very badly to being told

> he can't do something. That's pretty much what a safety authorities's job is. Any professional safety engineer would last

>  microseconds in that kind of environment.

Your assuming that’s the case but other than a possibly sensationalist biography we actually have no idea of how the company operates in detail. For example how does final design approval take place? It may be a whole lot less frenetic – we don’t know. One workable hypothesis is that if they find enough solutions, one will be the right one, they just have to get the culling process correct.

> 5. Musk has achieved order of magnitude reductions in cost by bringing much of the design in-house. This means he is probably

> using designers with little background in what they are designing (there are some examples of this in the book), including what

> constitutes an unsafe design. The zeitgeist is, the establishment is bloated with too much cash, old, uncool and don't know

> what they're doing. In contrast we are younger, we are smarter, we are cooler, we can do it better. There's probably an element

> of truth in this but when it comes to safety engineering which, let's face it, can be boring as bat shit, you have to wonder if they're

> missing or ignoring something.

I don’t think that the argument that an “established” organization is better because they can take their time/have processes/have experience holds that much water, case in point – this thread was started by an establishment organization (Chrysler/Fait) getting it really badly wrong. Then we can talk about li-ion batteries, Tesla so far appears to have done a slightly better job than – let’s say Boeing?

Heck the reason this list exists in the first place is that anyone and everyone seems to get it wrong. If we made a list who would be included/excluded?

We also have to remember that as an automotive manufacture they have to go though the same set of hoops and meet the same set of standards and regulations that every other automotive manufacture has to meet, in particular European type approval (probably a legal necessity as they manufacture or at least assemble in the Netherlands) and the independent testing generally considered necessary in the USA. The implication there is that ISO 9001 at least is required and probably conformance to ISO 26262 in some form

I don’t see anything in the argument that proves what Tesla is doing anything unsafe, however it does sound like a rather crappy environment to work for long periods of time, but saying that it  does sound rather exciting! I almost feel like I’m twenty something again contemplating working there :-) then the sound of the boys on school holiday reminds me I made other life choices.

Update: Given I wrote the above I went and did a Google search for “Tesla Motors ISO 9001” and found the following in TESLA’s website

“Nbk240. TM do have a ISO 9001:2008 certification. The new version ISO 9001:2015 is In the pipeline :) Most importantly, the religion of continuos Improvement included ! Another advice, pay strong attention to ISO 31000 Risk Management Standard :)”

LinkedIn also has several jobs listed for TESLA MOTORS for senior engineers, quality auditors etc. and there is a range of standards including ISO 9001 and ASQ and etc. included in the job requirements in the requirements along with X years experience. At a minimum they at least know the standards exist.

From: Les Chambers [mailto:les at chambers.com.au] 
Sent: 22 July 2015 22:52
To: 'Mike Ellims'; 'Matthew Squair'; 'Heath Raftery'
Cc: systemsafety at lists.techfak.uni-bielefeld.de
Subject: RE: [SystemSafety] Hackers take over *control* of a car wirelessly

Mike

I agree, it was a sweeping statement ... or more an impression gained from reading Ashlee Vance's biography of Elon Musk. I found Vance's account credible for the following reasons:

1. Vance researched it thoroughly. He interviewed many people from Musk's personal and professional life. Halfway through its writing Musk caved in and agreed to cooperate even though his requests to "write footnotes" and edit content were refused.

2. Vance bent over backwards to give a balanced account of Musk's adventures. There's a lot of good, some bad and much ugly in this story.

Given it was published for general consumption there is no detail on engineering process in Musk's companies, Space X and Tesla, but from a safety engineering point of view you can't help but gain the following impressions.

1. All Musk companies operated (and may still be operating) under the constant threat of bankruptcy (situation normal for a start-up). The temptation to take shortcuts would have been huge (at least in the early days – and possibly still today).

2. Eighty to one hundred hour work weeks are normal. New hires are warned that, "this is special forces territory". I tried this once for a year and discovered that you make dangerous design decisions late at night. After a year of this I turned into a gibbering idiot and took six months to recover. This leads me to my next point:

3. Musk burns out his people. His turnover is high. The only people with the stamina to sustain this workload are probably 20 and 30 somethings who are learning on-the-job. You wonder what kind of design decisions they are making at midnight.

4. Disagreeing with musk is career limiting, proving him wrong is fatal. He is not what you'd call a well adjusted kind of guy. Try establishing and running a consistent safety engineering regime in this kind of environment. Musk reacts very badly to being told he can't do something. That's pretty much what a safety authorities's job is. Any professional safety engineer would last microseconds in that kind of environment.

5. Musk has achieved order of magnitude reductions in cost by bringing much of the design in-house. This means he is probably using designers with little background in what they are designing (there are some examples of this in the book), including what constitutes an unsafe design. The zeitgeist is, the establishment is bloated with too much cash, old, uncool and don't know what they're doing. In contrast we are younger, we are smarter, we are cooler, we can do it better. There's probably an element of truth in this but when it comes to safety engineering which, let's face it, can be boring as bat shit, you have to wonder if they're missing or ignoring something. 

I'd recommend the book. It's a rattling good tale but it leaves you conflicted. On the one hand you can't help admiring Musk and what he has achieved in such a short time. He's probably a one in 100 year kind of guy. He's given the technology industry a good kick in the pants, created thousands of new jobs and moved space and road vehicle technology a few big steps into the future. Any capital invested in his companies has been very productive. It's also true that if you're going to make advances like this you've got a take risks. That is, drop your risk tolerance and be prepared to take casualties. 

That said I challenge you to read the book and ask yourself this question:

Would I want to be an astronaut sitting on top of one of Musk's rockets?

Right now MY answer would be NO! Taking casualties is fine as long as I'm not one of them. I'm content to be old and uncool. I'll leave that to the young and the fearless.

Cheers

Les

From: Mike Ellims [mailto:michael.ellims at tesco.net] 
Sent: Wednesday, July 22, 2015 7:07 PM
To: 'Les Chambers'; 'Matthew Squair'; 'Heath Raftery'
Cc: systemsafety at lists.techfak.uni-bielefeld.de
Subject: RE: [SystemSafety] Hackers take over *control* of a car wirelessly

Wow.

> Tesla motors employs mostly twenty somethings, if Elon Musk's biography is any indication, I suspect they don't have much adult supervision.

That’s a pretty sweeping statement and given the number of employees (10,000+) probably isn’t completely correct. Tesla has actually been around for a while so the 20 and 30 something’s that started are now 30 and 40 something’s. In addition it may not be that relevant as age is only an indication of experience not of ability. I suspect that what they actually have is la bunch of 30 and 40 something’s heading up department with lots of 20 and 30 something doing the work. I don’t know about you but I suspect I was at my best somewhere between 30 and 50, most of the drive of a twenty something and just enough mistakes to make me careful.

Tesla had its big security scare a few years back when the app interface got hacked and lead to a number of incidents with flashing lights and horns being beeps and unlocking of doors. Since then it seems to have upped its efforts on security. e.g.

http://cleantechnica.com/2014/02/18/tesla-motors-snags-kristin-paget-apple/

http://www.computerworld.com/article/2597937/security0/tesla-recruits-hackers-to-boost-vehicle-security.html

From: systemsafety-bounces at lists.techfak.uni-bielefeld.de [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Les Chambers
Sent: 22 July 2015 04:06
To: 'Matthew Squair'; 'Heath Raftery'
Cc: systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] Hackers take over *control* of a car wirelessly

Dynamic updates to motor-vehicle functionality are a fact of life now. Tesla Motors is a "world leader" with this. Here is the release note list for the Tesla model S:

http://www.teslamotorsclub.com/showwiki.php?title=Model+S+software+firmware+changelog

All you need is a wireless net to get the job done in around 45 minutes. Theoretically you could probably do it with a mobile phone hotspot. I don't know if the model S as to be stationary for the upgrade to occur. Hope so.

Ethical discussions on whether or not this is a good thing are probably irrelevant. It's just happening all around us. Tesla motors employs mostly twentysomethings, if Elon Musk's biography is any indication, I suspect they don't have much adult supervision. It reminds me of the Battle of the Bulge where Hitler put kids in tanks who had never seen a tank battle. They had little concept of what was coming and no fear.

So if you don't like this trend you're probably old and uncool. Why shouldn't an auto be just like a mobile phone or your Windows laptop? What could go wrong???

From: systemsafety-bounces at lists.techfak.uni-bielefeld.de [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Matthew Squair
Sent: Wednesday, July 22, 2015 11:14 AM
To: Heath Raftery
Cc: systemsafety at lists.techfak.uni-bielefeld.de
Subject: Re: [SystemSafety] Hackers take over *control* of a car wirelessly

If someone can seriously think that updating hospital drug pump firmware via the interwebz is a 'good idea' I think there's minimal likelihood of a good flogging in the town square happening anytime soon.

http://criticaluncertainties.com/2015/06/23/all-your-drug-pumps-are-belong-to-us/

Matthew Squair

MSysEng, MIEAust, CPEng

Mob: +61 488770655

Email; Mattsquair at gmail.com

Web: http://criticaluncertainties.com

On 22 Jul 2015, at 10:45 am, Heath Raftery <heath.raftery at restech.net.au> wrote:

On 22/07/2015 3:44 AM, Martyn Thomas wrote:

On 21/07/2015 18:27, Tom Ferrell wrote:

Stating the obvious, but isn’t there an aspect of this that goes

something like, “Just because we can doesn’t mean we should.” To me,

there is a fundamental engineering ethics question that comes into

play when people start talking about the ‘Internet of Everything.’

When someone postulates hooking two systems together that always

before have been physically separated, engineers have a moral

responsibility IMHO to inject themselves firmly and fully into the

benefits vs. risks discussion with a strong bias of when in doubt, don’t.

That sounds like excellent advice, but if I'm happy to connect A to B

and B to C, and you are happy to connect X to Y and Y to Z, whose fault

is it when Peter connects one of (A,B,C) to one of (X,Y,Z) and something

bad happens?

The general philosophical arguments are worth having, but doesn't this particular case offer a more direct argument?

If you're the one that connects cellular to CAN (via whatever paths already exist), you ought to be shot, stripped and jailed for gross negligence, *before* there's even an accident caused.

I'm flabbergasted that Chrysler could have released a vehicle where that electronic link even exists. No "great new feature"(TM) warrants such a gaping hole that would get every hacker from here to hell tapping away at the new door. There is zero evidence that anyone has ever designed a robust enough system that you could honestly connect the two and claim it safe.

All the "great new features" that are on the horizon can be achieved without making that link - updates over the air, Internet connected entertainment, vehicle location, etc. I see no excuse.

Heath

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE

  _____  

 <https://www.avast.com/antivirus> Image removed by sender. Avast logo

This email has been checked for viruses by Avast antivirus software. 
www.avast.com <https://www.avast.com/antivirus>  

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20150723/19b6ddbd/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20150723/19b6ddbd/attachment-0001.jpg>