[SystemSafety] UC The Accident to SpaceShip2

[Smith, Brian E. (ARC-TH)]

Some time ago (2008) the FAA published the following report: Development
of a Scoring Algorithm for Flight Crew Intervention Credit in System
Safety Assessments

See: http://www.tc.faa.gov/its/worldpac/techrpt/ar0845.pdf

I don¹t believe it was ever fully adopted.  Abstract:

	
		
		
	
	
		
			
				
					
						According to current regulations for type certification of large
commercial aircraft, certification credit may be taken for correct and
appropriate action for both quantitative and qualitative assessments
provided that some general criteria are fulfilled. According to the same
regulations, quantitative assessments of the probabilities of flight crew
errors are not considered feasible. As a consequence, the system designer
is allowed to take 100% credit for correct flight crew action in response
to a failure. Earlier research has indicated that this leads to an
overestimation of flight crew performance.

						The overall goal of this research effort was the development of a
method that would allow certification credit for good human factors design
practice in certification regulation. This method consists of a scoring
algorithm that combines key flight deck design characteristics into an
overall level of certification credit for flight crew intervention in the
case of system failures.

						The method is easy to apply, provided that the system failure modes
are associated flight deck annunciations are known [a big IF, in my
opinion]. As expected, application of the method to a number of example
cases shows that it differentiates between system failures and between
aircraft types. The method also produces higher average scores for more
modern cockpits.

						Although every possible effort was spent in making this a valid,
practicable, and acceptable method, it is still the result of a research
project. Further development is recommended.


Brian


On 8/5/15, 7:08 AM, "systemsafety-bounces at lists.techfak.uni-bielefeld.de
on behalf of Corkery Tony"
<systemsafety-bounces at lists.techfak.uni-bielefeld.de on behalf of
TCORKERY at qinetiq.com> wrote:

>I too found the implied censure a little harsh.  The specific criticism
>was that they had not properly considered the effect of human errors;
>pointing to AC 437.55-1 where it does indeed cover the areas involved in
>the crash.  However, the response from the engineers is that they had
>considered it but concluded that the pilot wouldn't do that action so
>didn't include any further safe guards.
>
>Engineers conduct their activities in the environment of regulation,
>guidance and their experience and whilst 437.55-1 is explicit in that
>requirement, I suspect the engineers involved would have been experienced
>in the civil aviation world where the AMC for 1309 states under
>'Operational and maintenance considerations' that for flight crew and
>maintenance tasks -
>
>'These tasks, which are related to compliance, should be appropriate and
>reasonable. Quantitative assessments of the probabilities of flight crew
>and maintenance errors are not considered feasible.  Reasonable tasks are
>those for which full credit can be taken because the flight crew or
>ground crew can realistically be anticipated to perform them correctly
>when they are required or scheduled.  For the purposes of quantitative
>analysis,* a probability of one* can be assumed for flight crew and
>maintenance tasks that have been evaluated and found to be reasonable...'
>
>So, having considered the task of operating the feather control at the
>correct time to be appropriate and reasonable, why (in the context of
>this guidance), would they consider any additional mitigation when they
>probably believed it was reasonable (and accepted practice) to assume a
>probability of 1 for compliance under internationally recognised guidance.
>
>This guidance has often given me concerns, in respect of maintenance
>tasks.  It is acknowledged that maintenance is a major contributor to
>accidents, yet application of this principle could mean that there is no
>real pressure to design systems to minimise maintenance intervention or
>spend effort making it simpler from a safety perspective, as you can
>define what you want, deem it appropriate and reasonable and claim a
>probability of 1 that it will be done correctly.
>
>Seems to be conflicting advice in existence?
>
>Cheers 
>Tony Corkery
>
>-----Original Message-----
>From: systemsafety-bounces at lists.techfak.uni-bielefeld.de
>[mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of
>Peter Bernard Ladkin
>Sent: 03 August 2015 13:15
>To: The System Safety List
>Subject: [SystemSafety] The Accident to SpaceShip2
>
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>
>The NTSB held its public hearing on July 28th. All infos, including
>presentations from the hearing, available at
>http://www.ntsb.gov/news/events/Pages/2015_spaceship2_BMG.aspx and the
>NTSB's provisional executive summary, findings and safety recommendations
>at 
>http://www.ntsb.gov/news/events/Documents/2015_spaceship2_BMG_abstract.pdf
>
>The NTSB is big on the HazAn not having dealt adequately with HF aspects,
>including that the accident showed there was a critical system (the
>feather actuation/stow/lock/mechanism) with a single point of failure,
>namely human error.
>
>However, I strongly disagree with the "summary" of Alister Macintyre, who
>wrote about it in the Risks Forum
>http://catless.ncl.ac.uk/Risks/28.83.html#subj1 He speaks about
>"cut[ting] corners", and writes as if he thinks various people did things
>wrong. I don't see much evidence for that at all (although it is possible
>that some might come with the full report). I see people trying to get a
>job done, to bring a highly innovative piece of critical engineering -
>pioneering is an apt word - to fruition. And in this largely novel
>environment, needing to improve their HazAn. The HazAn is likely
>substantial intellectual property. Without evidence, it's on the verge of
>insulting to suggest anyone or any group involved with this project was
>slacking.
>
>Compare. Lithium-ion primary and auxiliary batteries on the Boeing 787 is
>also new technology. An FMEA was done that suggested the worst that could
>happen to the environment during thermal runaway of one or more cells was
>development of smoke. That FMEA remained unchanged even after a
>thermal-runaway event during testing burnt down the test facility. And
>the NTSB visited the fabricating factory where it observed that hazard
>mitigation, namely certain quality control measures, was not as effective
>as was thought 
>http://www.ntsb.gov/investigations/AccidentReports/Reports/AIR1401.pdf .
>Boeing has a lot more at stake - maybe the entire company again, who
>knows? - in getting it right than the backers of Scaled Composites. And
>they still didn't get the HazAn right.
>
>When the technology is new, HazAn is a tricky business. No one wants to
>get it wrong. But they do.
>And they will. Which is why some of us are working on ways to get it done
>better.
>
>I say more at 
>http://www.abnormaldistribution.org/2015/08/03/the-accident-to-spaceship-t
>wo/
>
>PBL
>
>Prof. Peter Bernard Ladkin, Faculty of Technology, University of
>Bielefeld, 33594 Bielefeld, Germany Je suis Charlie
>Tel+msg +49 (0)521 880 7319  www.rvs.uni-bielefeld.de
>
>
>
>
>-----BEGIN PGP SIGNATURE-----
>
>iQEcBAEBCAAGBQJVv1s5AAoJEIZIHiXiz9k+TeUIAIQJFdC4U8GaTy/dp5Mc2o1i
>43sQH6wtT0sCNDjGPGAeQtSYrqyfIyPnw8WJmUY4ZBHfJlLnlN0gkeR5f41/kK6T
>WI/w1HzHuRX6vWtOIMkYHPmwm5c58frNFsDMu6/R+Egv21DnPy7qhVN4pajsNpPX
>DwSselt2SiHD0ELd8SEfUgkALjYzfLNDIo9JKEVw8QgXinRHJqVPxeZsITHxBT1X
>2YBdcsK3tpRB135yIAqYABsgE9Qe2aO3jQTwFi/3DPNG9EWSqqp8bjmFulDRYXtp
>/nFoXJG9uX0LAKOwGqEQlK8UzYZotEa2GzkB1DK3ORBr+9lV+8vk5oGLvr/ibW0=
>=JhzT
>-----END PGP SIGNATURE-----
>_______________________________________________
>The System Safety Mailing List
>systemsafety at TechFak.Uni-Bielefeld.DE
>This email and any attachments to it may be confidential and are
>intended solely for the use of the individual to whom it is
>addressed. If you are not the intended recipient of this email,
>you must neither take any action based upon its contents, nor
>copy or show it to anyone. Please contact the sender if you
>believe you have received this email in error. QinetiQ may
>monitor email traffic data and also the content of email for
>the purposes of security. QinetiQ Limited (Registered in England
>& Wales: Company Number: 3796233) Registered office: Cody Technology
>Park, Ively Road, Farnborough, Hampshire, GU14 0LX
>http://www.qinetiq.com.
>_______________________________________________
>The System Safety Mailing List
>systemsafety at TechFak.Uni-Bielefeld.DE