[SystemSafety] Thorough Test Suite?

Tue Nov 19 17:06:26 CET 2019

Chris Hills wrote:

>> I think that if C were taught as Software Engineering, and a discipline,

>> the defect rate of C would tend towards that of Ada.

I think this is conjecture… C is a flawed specified language – as with most flawed specifications, the only real way of resolving issues is to fix those issues.

You can only polish a turd so much…

A

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Chris Hills
Sent: 18 November 2019 12:21
To: 'Mike Ellims'; 'Dewi Daniels'; 'Roderick Chapman'
Cc: 'The System Safety List'
Subject: Re: [SystemSafety] Thorough Test Suite?

This is something I have often commented on. 

Ada is taught as part of Software Engineering  for reliable/high integrity systems

C is taught as part of programming…… (and has the ethos of  “trust the programmer” )

The whole ethos, especially in the teaching and using, between Ada and C is different.   I don’t know any hackers that use Ada.

I think that if C were taught as Software Engineering, and a discipline,  the defect rate of C would tend towards that of Ada. How far it will tend to Ada’s defect rate  is up for discussion in the bar. 

I  recall some data on this in Les Hatton’s Safer C  (still(?) available in all good bookshops) that showed there was a wide range in the defect rate of C.  Using good Software Engineering discipline, methods and tools e.g. static analysis, subsets, style guides etc.  the defect rate was greatly reduced. 

So, yes, if you get better trained C software engineers things will improve.  However if it will be as good as Ada (let alone SPARK)  or just “good enough” is a different debate.  We can move to a different bar for that one. J 

Regards

   Chris 

Phaedrus Systems Ltd         

FREEphone 0808 1800 358    International +44 1827 259 546
Vat GB860621831  Co Reg #04120771
Http://www.phaedsys.com <http://www.phaedsys.com/>   chills at phaedsys.com 

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Mike Ellims
Sent: Sunday, November 17, 2019 9:58 AM
To: 'Dewi Daniels'; 'Roderick Chapman'
Cc: 'The System Safety List'
Subject: Re: [SystemSafety] Thorough Test Suite?

One question that this doesn’t address is of course whether the SPARK ADA code was better than the c code because SPARK is better or the programmers writing the ADA were better...

Damn those confounding factors...

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Dewi Daniels
Sent: 17 November 2019 08:59
To: Roderick Chapman
Cc: The System Safety List
Subject: Re: [SystemSafety] Thorough Test Suite?

Rod,

About half the software was analysed by Lloyd's Register, the other half was analysed by Aerosystems. We only looked at the Level A and Level B code because it would have been too expensive to analyse all of the software on the aircraft. I was one of the team leaders at Lloyd's Register, so I was personally involved in conducting some of the static analysis and I was also aware of the work being done by other Lloyd's Register teams and by Aerosystems. It may be that some of the software might have been through the full DO-178B verification process, but that was not the case for the majority of the software. I personally analysed the worst C program cited in the article, which had an anomaly rate of 500 anomalies per thousand lines of code. The design didn't match the requirements and the code didn't match the design.

I remember that the mission computer software was written in SPARK. That program had the lowest anomaly rate of any of the programs that were analysed. The reported anomaly rate was four anomalies per thousand lines of code. As you know, Praxis typically achieved a residual defect rate of less than one defect per thouand lines of code for software developed using a similar process.

Yours,

Dewi Daniels | Director | Software Safety Limited

Telephone +44 7968 837742 | Email d <mailto:ddaniels at verocel.com> ewi.daniels at software-safety.com

Software Safety Limited is a company registered in England and Wales. Company number: 9390590. Registered office: Fairfield, 30F Bratton Road, West Ashton, Trowbridge, United Kingdom BA14 6AZ

On Fri, 15 Nov 2019 at 11:31, Roderick Chapman <rod at proteancode.com> wrote:

On 14/11/2019 17:11, Dewi Daniels wrote:

I helped conduct the static analysis on C-130J when I was at Lloyd's Register. QinetiQ's analysis is flawed. Due to timescale pressures, we were asked to conduct the static analysis before the code had been tested, so I don't see that you can draw any conclusions about the efficacy (or otherwise) of the DO-178B verification process.

Dewi,

 When you say "before the code had been tested", do you mean _all_ the code, or only the subsystems that L-R and you personally looked at? Were those systems the Level-A and Level-B systems that German compared wrt the efficacy of MC/DC structural coverage?

(Note: I also had a hand in this: a team from Praxis worked on the Mission Computer development at L-M in late 1995, during the phase when they adopted SPARK. I recall the experience fondly.)

 - Rod

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE
Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety

This email has been scanned by BullGuard antivirus protection.

For more info visit www.bullguard.com <http://www.bullguard.com/tracking.aspx?affiliate=bullguard&buyaffiliate=smtp&url=/> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20191119/b0252f5a/attachment-0001.html>