[SystemSafety] OpenSSL Bug

[Patrick Graydon]

On 15 Apr 2014, at 14:37, David Crocker <dcrocker at eschertech.com> wrote:

> Regarding the use of language subsets to reduce the occurrence of coding
> errors in C and C++, I published a short article in the January 2009
> edition of the SCSC newsletter called "Some Common C/C++ Coding Errors
> and Their Occurrence Rates" (available via http://scsc.org.uk/p106).
> This gave figures for densities of some types of coding error found in a
> large body of production C++ code having many different authors, based
> on a very shallow static analysis of the code. All these coding errors
> would violate at least one MISRA rule. This is evidence that to me at
> least that *some* of the MISRA rules are valuable in reducing coding
> errors.

This sort of analysis seems like a good basis for a draft of a standard.  But a correlation between defects and a construct does not necessarily show that a particular rule eliminating that construct will lower the rate of defects overall.  The rule might, for example, push programmers into using a different, even-more-error-prone construct.  Humans are so damn unpredictable.

In any case, turning to your article, the first problem you identify is the old = vs == one.  You associate this with rules 12.6 and 13.1 (total prohibition of assignments in conditionals).  Boogerd shows a negative correlation for both.  

The next up is use-before-initialisation, rule 9.1.  Boogerd shows no correlation for this one. 

The third is bitwise operator precedence, rule 12.1.  Boogerd shows a positive correlation for this one.

The fourth is type conversions, rules 10.1 and 10.2.  Boogerd observed no violations of 10.2 (and so could not comment on it) but a positive correlation for rule 10.1.

As Hatton, Boogerd, Moonen, Jones, and probably others point out, some rules work well in some cases and others don’t.

Neither your article nor theirs includes details that would explain why, for example, = vs == was a big problem in the code you analysed but negatively correlated for them.  We can only speculate.  For example, I wonder whether this has to do with how prevalent basic static analysis has become in C/C++ compilers.  Most compilers seem to check for assignments being used as though they were Booleans rather than for *any* use of assignment in a conditional.  A programmer who writes a lot of checks of the form ‘if ((r = check()) != SUCCESS) …’ might generate *loads* of violations of 13.1.  Yet, since many compilers would complain about ‘if ((r = check()) = SUCCESS)’, that programmer might also make no more that the usual number of errors on these lines (e.g. due to calling the wrong function, passing the wrong parameters, etc.).  We don’t really know why they found what they did, only that, for at least one program, violations of MISRA-C’s total ban on assignments in conditionals did not predict defectiveness.

Incidentally, I’ve been begging industry colleagues for access to their code and bug records so I can get an MS student to replicate the Boogerd study on a safety-critical C code base.  So far no takers.  And here I am thinking it would be nice to have more solid evidence about what works and what doesn’t.

— Patrick