[SystemSafety] Qualifying SW as "proven in use" [Measuring Software]

[Derek M Jones]

Peter,

> Getting this evidence is pretty tricky, as parallel developments for the
> same project won't happen.

NASA spends hundreds of millions to take a few snaps of a distant
planet, we need software engineering researchers with some vision
and drive (and connections).

Lets say $1million for a project, run this 20 times in parallel
to get some statistical significance and add in say $5 million to
cover the research side.  $25 million, peanuts ;-)

Of course NASA funding is really a way of channeling money to
political constituencies.  It might just happen that the random
selection of those 20 project locations appear to follow the
NASA pattern, not a problem.

For those who currently don't have $25 million in funding...
The SQALE document, along with many other documents of this kind,
list rules of the form "don't use XYZ".
The XYZ listed by SQALE are among the usual suspects.

If developers cannot use XYZ then they will have to use some other
construct.  I have yet to see any analysis that compares a XYZ
against the alternatives (I work in source code analysis and keep my
eyes open for such work).

For a lot less than $25 million an experiment comparing developers
allowed to use a particular XYZ vs. those not allowed to use XYZ
could be run.

It might be the case that while XYZ is bad it is actually better
than the alternatives.


> But you might be able to infer something on average over multiple projects.
>
> Derek M Jones wrote:
>> Thierry,
>>
>>> To answer your questions:
>>> 1°) Yes, there is some objective evidence that there is a correlation
>>> between a low SQALE index and quality code.
>>
>> How is the quality of code measured?
>>
>> Below you say that SQALE DEFINES what is "good quality" code.
>> In this case it is to be expected that a strong correlation will exist
>> between a low SQALE index and its own definition of quality.
>>
>>> For example ITRIS has conducted a study where the "good quality" code
>>> is statistically linked to a lower SQALE index, for industrial
>>> software actually used in operations.
>>
>> Again how is quality measured?
>>
>>> No, there is not enough evidence, we wish there would be more people
>>> working on getting the evidence.
>>
>> Is there any evidence apart from SQALE correlating with its own
>> measures?
>>
>> This is a general problem, lots of researchers create their own
>> definition of quality and don't show a causal connection to external
>> attributes such as faults or subsequent costs.
>>
>> Without running parallel development efforts that
>> follow/don't follow the guidelines it is difficult to see how
>> reliable data can be obtained.
>>
>

-- 
Derek M. Jones                  tel: +44 (0) 1252 520 667
Knowledge Software Ltd          blog:shape-of-code.coding-guidelines.com
Software analysis               http://www.knosof.co.uk