[SystemSafety] A small taste of what we're up against

[Derek M Jones]

Thierry,

> 1°) I wish too we were not constrained by all the confidentiality agreements and proprietary information, so I second Derek's claim on hard evidence. However, since I am bound by said agreements, I can only state generalities, and refer the ladies and gentlemen in this list to their own observations and data collecting.

Confidentiality and nondisclosure agreement do slow down wide-spread
data analysis.

Researchers in business schools have done some great work with
confidential data:
http://shape-of-code.coding-guidelines.com/2018/09/11/business-school-research-in-software-engineering-is-some-of-the-best/

> In fact, setting up a sound data collecting process is quite hard, and Jean-Louis Letouzey and I have work hard to identify standard defects in software, from requirements to implementation to testing, that actually do impact software reliability and software technical debt, and publish a free database that is language-agnostic. It is on purpose built as to be able to compare software built using several technologies and be comparable across technologies, using measurement theory. It is now for the industry to take this up.  We have noticed the usual tendency of the tool makers to want to introduce their "own" additions and sell their tools at and advantage which is natural. I hope more people will publish their evidence and hard data.

Collecting data is very hard and many such efforts have a short
lifetime.  The collection costs money and obviously managers
want to see a return on this investment.  So the collected data
has to pay its way (hopefully by providing guidance about what is
going on).

> In my experience (I can't write more) Inspections and peer reviews find fewer defects of the kind that automatic tools can find, and yet, inspections and peer reviews find defects that tools just cannot find.

Yes, people can approach the code at a higher level of abstraction
and see different kinds of problems.

There is lots of talk about AI solving this problem.  If the AI
was that good, why are people writing the code in the first place?

http://shape-of-code.coding-guidelines.com/2009/11/27/software-maintenance-via-genetic-programming/

-- 
Derek M. Jones           Software analysis
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com