[SystemSafety] How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?

Tue Jun 14 14:11:57 CEST 2016

Moring Thierry,

> accidents on the normal roads seem to be 4x times more likely to happen than on the motorways. Is that a symptom of bad driving or is that a symptom of bad infrastructure?

>From the point of view of infrastructure motorways are much safer than other roads for a number of reasons, mostly due to design.

-  motorways in general have the widest lanes thus there is more room to take avoiding action,

- likewise IF people are playing by the rules then they also have the largest inter-vehicle gaps so again there is more room for drivers to take avoiding action.

-  they tend to have enough area at the side of the road so that vehicles can in general be moved safety off in case of trouble e.g. breakdown.

Perhaps more importantly they are divided roads i.e. drivers can ignore oncoming traffic and mostly the traffic is all moving at about the same speed in the same direction with the exception of large vehicles which are all limited to 56 mph. This results in fewer vehicle to vehicle conflicts (passings). Also there are usually no exceptionally slow vehicles and pedestrians, cyclists, horse etc. are all absent (or should be)all of  which probably reduces the drivers cognitive load and definitely reduces the severity which in turn means that the number of accidents are probably under reported (police don’t usually attend fender benders unless it blocks the road).

As an example of some of these factors was when I was involved testing a tractor unit on I96 in Detroit one rainy day (it would ended very different if it wasn’t raining). Someone tried to overtake the tractor unit just as the on-ramp we were on narrowed from two lanes to one. The rear of the overtaking car was clipped by the front of the truck (they pulled in too some) and the car was swung around onto the front bumper of the truck; and was for a short time being pushed sideways. When we braked the truck,  car slide off onto I96, spun across 5 lanes of traffic and stopped on the centre divider. The driver of the car being pushed had zero control but no other vehicle collided with it as it spun as they had sufficient room to either break or steer around as did following vehicles. The driver of the car was unhurt and their most dangerous moment was attempting to walk from the barrier back to the side of the road. Here the artificial nature of the road gave the humans sufficient leeway to be able to cope.

Outside of motorways the road system is much more “busy” and diverse, with significantly more visual clutter, more conflicts e.g. oncoming traffic, cars pulling out of side stress and driveways, cars stopped in various locations, people, horses, narrow lanes overhanging trees etc. etc. etc. So the cognitive load is much higher all drivers and surprise becomes a bigger factor AND there are fewer options for everyone. Consider that in many cases the roads in the UK were originally laid out for horses and carts and motorized traffic has been squeezed onto them.

Thus for motorways the nature of the infrastructure dominates but this constrains both the types of vehicle interactions that can take place and the responses that drivers can/will take.

> Would the same happen for autonomous vehicles?

Yes I think so, I suspect that Tesla and everyone else will get very good at navigating highways but it will take a lot longer before it can cope with everything.

> When Tesla limits the use of their software to highways, is it the same symptom?

I suspect it’s the same thing i.e. it can be done on motorways because the environment is sufficiently constrained and artificial compared to the majority of other roads. An import point is that there are no people not in cars; people are hard to detect relative to metal boxes; in general they aren’t good radar reflectors

> Does it mean than on a badly designed infrastructure, the humans are more able to prevent accidents from happening than an AV? 

Possibly, possibly not; humans get distracted (phones, kids in the car looking for a house, talking one observations study done in London showed that around 20% drivers could be classed as distracted), humans probably have a lower situational awareness in many cases e.g. can’t look both ways at a junction at the same time and so on. Humans are however possibly better at anticipating the reactions of other people e.g. seeing that the wobbly kid riding a scooter on the footpath might suddenly come into the road.

> It seems most literature is only looking at the “bad” effects of human intervention instead of also looking at the good effects too.

In general yes as most data comes from looking at accidents that do occur though there is a large body of information from human factors looking at what drivers actually do in emergencies  and it’s no very pretty e.g. 40% driver never brake in an emergency and the majority don’t brake hard enough. However partial actions can reduce the level of severity… some braking is better than no braking and a side swipe is better than a head on. Currently it’s not clear if systems such as emergency brake assist do much better than people but ESC (electronic stability control) can be shown to before much better than humans at avoiding accidents.

One way in which the human factor is taken into account is the method used to determine risk in ISO 26262 in that severity is used as one of the three inputs (severity, exposure, controllability). If severity based on statistical data i.e. actual road accidents (as hinted at in the standard) then the avoiding actions  taken by drivers and other road users is baked into the severity ranking.

> In the future, if the roads were designed for AV, would the safety increase for the AV and decrease for humans, or increase for both?

Possibly a moot point, it’s not going to happen any time soon… for the same reason hydrogen fuel cell cars aren’t going to happen any time soon. The cost is just too high. To some degree you the highways authorities can even maintain the infrastructure that exists e.g. road markings that haven’t been repainted for years signs and have faded signs that are obscured by plant growth etc. etc. There are 2,459,000 miles at the end of 2015 if it cost £100 per mile (low motor ways are 1 million per mile) to upgrade specifically for AV’s it’s £200 million not a huge amount but keeping road markings fresh and signs clear would help both HV’s and AV’s.

As a design principle if roads were modified for AV’s then we should expect that they are at least as safe for HV’s.

> Is there literature on this subject?

Yes but very wide and not very deep and quite dispersed. There are several journals devoted to traffic and accident safety as well and quite a lot of work gets published as human factors e.g. Human Factors seems to devote at least one issue per year. There are also several books but again very broad and they tend to focus on either design or on accidents and issues such as drink driving.

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Coq, Thierry
Sent: 14 June 2016 10:18
To: 'Bielefield Safety List'
Subject: Re: [SystemSafety] How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?

I wonder, in a systems analysis point of view, how much of the safety is attributable to the human drivers and how much to other factors, such as the infrastructure.

Accidents seem to be clumped, whenever there is a map available. Some roads are particularly prone to traffic accidents. Are these well-designed roads?

For example, accidents on the normal roads seem to be 4x times more likely to happen than on the motorways. Is that a symptom of bad driving or is that a symptom of bad infrastructure? A 4x increase would mean for any systems engineer that the infrastructure is the dominating facture for safety, not the humans.

Would the same happen for autonomous vehicles? When Tesla limits the use of their software to highways, is it the same symptom?

Does it mean than on a badly designed infrastructure, the humans are more able to prevent accidents from happening than an AV? 

(it seems most literature is only looking at the “bad” effects of human intervention instead of also looking at the good effects too)

In the future, if the roads were designed for AV, would the safety increase for the AV and decrease for humans, or increase for both?

Is there literature on this subject? 

Best regards,
Thierry Coq

 <http://www.dnvgl.com/> www.dnvgl.com  

The statements expressed here are my own, not those of my employer.

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Mike Ellims
Sent: lundi 13 juin 2016 00:44
To: martyn at thomas-associates.co.uk
Cc: 'Bielefield Safety List'
Subject: Re: [SystemSafety] How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?

[…]

Possibly, but the safety of the system in total is dominated by the number of errors that human drivers make. If AV’s are in general significantly safer than human drivers (cause fewer accidents and/or accident severity is lower) then is the case to be made that people shouldn’t be allowed to drive?

[…]

**************************************************************************************
This e-mail and any attachments thereto may contain confidential information and/or information protected by intellectual property rights for the exclusive attention of the intended addressees named above. If you have received this transmission in error, please immediately notify the sender by return e-mail and delete this message and its attachments. Unauthorized use, copying or further full or partial distribution of this e-mail or its contents is prohibited.
**************************************************************************************

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20160614/e9e70aec/attachment-0001.html>