[SystemSafety] MC/DC coverage assumptions

[Michael Holloway]

​My attempt to send the reply below from my NASA account seems to have
failed silently. Perhaps this will work.​


​I​
 c
​
oncur with Dewi's comments on the specific example.

The definitive discussion about what constitutes MC/DC* is found in Section
4.13 of DO-248C. If the sponsoring organization for the DO series of
documents had a rational interpretation of 'fair use' of copyrighted
material, I'd include text from that section here. But they don't, so I
won't.  It would've been possible to clear up the confusion caused by the
poor descriptions and 'definitions' associated with MC/DC in DO-178, but
the committee chose to improve the explanatory material in 248C instead.

* The phrase 'MC/DC coverage' (which is 'Modified Condition / Decision
Coverage coverage') should never be used. It is as gauche as 'PIN number'.

-- 
All the best,
*C. Michael Holloway* (cMh)
Senior Research Computer Engineer
NASA Langley Research Center, Hampton VA USA
bit.ly/cmhpubs

Verba volant, scripta manent
spoken words fly away, written words remain

(The words in this message are mine alone;
neither blame nor credit NASA for them.)


On Wed, Feb 28, 2018 at 10:21 AM, Dewi Daniels <
dewi.daniels at software-safety.com> wrote:

> Derek,
>
> The definition of Modified Condition/Decision Coverage is:
>
> "Every point of entry and exit in the program has been invoked at least
> once, every condition in a decision in the program has taken all possible
> outcomes at least once, every decision in the program has taken all
> possible outcomes at least once, and each condition in a decision has been
> shown to independently affect that decision's outcome. A condition is shown
> to independently affect a decision's outcome by: (1) varying just that
> condition while holding fixed all other possible conditions, or (2) varying
> just that condition while holding fixed all other possible conditions that
> could affect the outcome".
>
> In your example, the single condition would have to have been evaluated to
> both true and false to achieve MC/DC. This means that at least two test
> cases are required to achieve either MC/DC or branch coverage, but one test
> case would be sufficient to achieve statement coverage.
>
> I'd be astonished if a tool with the pedigree of LDRA Testbed got this
> wrong.
>
> Dewi
>
> Yours,
>
> Dewi Daniels | Director | Software Safety Limited
>
> Telephone +44 7968 837742 <+44%207968%20837742> | Email d
> <ddaniels at verocel.com>ewi.daniels at software-safety.com
>
> Software Safety Limited is a company registered in England and Wales.
> Company number: 9390590. Registered office: Fairfield, 30F Bratton Road,
> West Ashton, Trowbridge, United Kingdom BA14 6AZ
>
> On 28 February 2018 at 14:35, Derek M Jones <derek at knosof.co.uk> wrote:
>
>> All,
>>
>> I was recently reading a paper that compared unit testing of
>> industrial embedded software with some open source programs.
>> The comparison included a table of statement, branch and MC/DC coverage,
>> items in the table included: aerospace software, automotive software and
>> subway signal software
>>
>> The MC/DC coverage numbers were a lot better than the statement and
>> branch coverage.  This is obviously a mistake, at best they can be
>> as good as.
>>
>> I emailed the authors, who have been very prompt replying.
>> The latest reply was a bit surprising.
>>
>> The algorithm they used for MC/DC assumes that a function containing
>> a single branch (e.g., an if-statement with no else part) and
>> the test involves a single condition (i.e., no AND or OR conditions),
>> then 100% MC/DC coverage is assumed, even if 100% branch coverage is
>> not obtained.
>>
>> Sounds like a mistake in their algorithm.  However, they claim there is
>> some amount of existing practice and even call out Testbed as
>> behaving like this (I don't have a copy to check this out).
>>
>> Somebody please tell me that this is not an assumption made by
>> commercial packages when calculating MC/DC coverage.
>>
>> The authors admit that MC/DC coverage cannot be better than
>> statement and branch coverage, and admit the current presentation
>> of MC/DC coverage in the table could be misleading.  They are going
>> to release a version with corrected data.
>>
>> --
>> Derek M. Jones           Software analysis
>> tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com
>> _______________________________________________
>> The System Safety Mailing List
>> systemsafety at TechFak.Uni-Bielefeld.DE
>>
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180228/606f0e94/attachment.html>