[SystemSafety] Thorough Test Suite?

[Dewi Daniels]

Martyn,

I assume the Boscombe Down analysis you're referring to is Andy German's
CrossTalk article on Lockeed C-130J?

I helped conduct the static analysis on C-130J when I was at Lloyd's
Register. QinetiQ's analysis is flawed. Due to timescale pressures, we were
asked to conduct the static analysis before the code had been tested, so I
don't see that you can draw any conclusions about the efficacy (or
otherwise) of the DO-178B verification process. Also, we reported every
defect we found, from defects that contributed to a hazardous failure
condition to spelling mistakes in documentation.

I personally conducted the static analysis of the worst program cited in
Andy German's paper (two software lines of code per anomaly). It never
stood any chance of getting through a DO-178B verification process. The
design didn't match the requirements and the code didn't match the design.
The software was rejected by the Designated Engineering Representative
(DER) and Lockheed eventually replaced it with software from a different
supplier.

The best program (250 lines of code per anomaly) was specified formally
using Parnas Tables, was written in SPARK and verified formally using the
SPADE Automatic Simplifer and the SPADE Proof Checker.

Yours,

Dewi Daniels | Director | Software Safety Limited

Telephone +44 7968 837742 | Email d <ddaniels at verocel.com>
ewi.daniels at software-safety.com

Software Safety Limited is a company registered in England and Wales.
Company number: 9390590. Registered office: Fairfield, 30F Bratton Road,
West Ashton, Trowbridge, United Kingdom BA14 6AZ


On Thu, 14 Nov 2019 at 16:31, Martyn Thomas <martyn at thomas-associates.co.uk>
wrote:

> Derek
>
> What's a "thorough test suite" and how would you recognise one? We know
> from theory and from a Boscome Down analysis that even MCDC testing can
> leave very high defect densities.
>
> Martyn:
>
> > A common research use (not much used in industry) is using mutated
> > code to check the quality of a test suite, i.e., a thorough test
> > suite will detect the added mistake.
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> Manage your subscription:
> https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20191114/bf5b6251/attachment-0001.html>