[SystemSafety] MC/DC coverage assumptions

[Derek M Jones]

Dewi,

> I don't think that's what Ian is saying at all. There are structural
> coverage tools that measure MC/DC as an optional extra step. Some of those
> tools only consider decisions with multiple conditions in their separate
> MC/DC analysis, as decisions with single conditions will have already been
> addressed by the decision coverage analysis, so there's nothing more to

This sounds like bad user interface design.  Users could be
led to believe the numbers are for MC/DC, when in fact they need
to merge in other values to get this data.

> coverage analysis. It sounds as if the authors of the paper didn't know how
> to drive the tool they were using.

Probably some junior guy who is now having a very bad week, and seven
co-authors with red faces (how often do you see lots of 100% MC/DC?)

They are either going to have to rerun all the data gathering again
or do major surgery on the paper.

> 
> Yours,
> 
> Dewi Daniels | Director | Software Safety Limited
> 
> Telephone +44 7968 837742 | Email d <ddaniels at verocel.com>
> ewi.daniels at software-safety.com
> 
> Software Safety Limited is a company registered in England and Wales.
> Company number: 9390590. Registered office: Fairfield, 30F Bratton Road,
> West Ashton, Trowbridge, United Kingdom BA14 6AZ
> 
> On 28 February 2018 at 15:56, Derek M Jones <derek at knosof.co.uk> wrote:
> 
>> Dewi,
>>
>> The definition of Modified Condition/Decision Coverage is:
>>>
>> ...
>>
>>> In your example, the single condition would have to have been evaluated to
>>> both true and false to achieve MC/DC. This means that at least two test
>>> cases are required to achieve either MC/DC or branch coverage, but one
>>> test
>>> case would be sufficient to achieve statement coverage.
>>>
>>
>> As Ian Broster's interesting reply shows, there are people out there who
>> have other ideas.
>>
>> I'd be astonished if a tool with the pedigree of LDRA Testbed got this
>>> wrong.
>>>
>>
>> As would I.
>>
>> Perhaps there is a compatibility flag, -do_what_broken_tool_does?
>>
>>
>>> Dewi
>>>
>>> Yours,
>>>
>>> Dewi Daniels | Director | Software Safety Limited
>>>
>>> Telephone +44 7968 837742 | Email d <ddaniels at verocel.com>
>>>
>>> ewi.daniels at software-safety.com
>>>
>>> Software Safety Limited is a company registered in England and Wales.
>>> Company number: 9390590. Registered office: Fairfield, 30F Bratton Road,
>>> West Ashton, Trowbridge, United Kingdom BA14 6AZ
>>>
>>> On 28 February 2018 at 14:35, Derek M Jones <derek at knosof.co.uk> wrote:
>>>
>>> All,
>>>>
>>>> I was recently reading a paper that compared unit testing of
>>>> industrial embedded software with some open source programs.
>>>> The comparison included a table of statement, branch and MC/DC coverage,
>>>> items in the table included: aerospace software, automotive software and
>>>> subway signal software
>>>>
>>>> The MC/DC coverage numbers were a lot better than the statement and
>>>> branch coverage.  This is obviously a mistake, at best they can be
>>>> as good as.
>>>>
>>>> I emailed the authors, who have been very prompt replying.
>>>> The latest reply was a bit surprising.
>>>>
>>>> The algorithm they used for MC/DC assumes that a function containing
>>>> a single branch (e.g., an if-statement with no else part) and
>>>> the test involves a single condition (i.e., no AND or OR conditions),
>>>> then 100% MC/DC coverage is assumed, even if 100% branch coverage is
>>>> not obtained.
>>>>
>>>> Sounds like a mistake in their algorithm.  However, they claim there is
>>>> some amount of existing practice and even call out Testbed as
>>>> behaving like this (I don't have a copy to check this out).
>>>>
>>>> Somebody please tell me that this is not an assumption made by
>>>> commercial packages when calculating MC/DC coverage.
>>>>
>>>> The authors admit that MC/DC coverage cannot be better than
>>>> statement and branch coverage, and admit the current presentation
>>>> of MC/DC coverage in the table could be misleading.  They are going
>>>> to release a version with corrected data.
>>>>
>>>> --
>>>> Derek M. Jones           Software analysis
>>>> tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com
>>>> _______________________________________________
>>>> The System Safety Mailing List
>>>> systemsafety at TechFak.Uni-Bielefeld.DE
>>>>
>>>>
>>>
>> --
>> Derek M. Jones           Software analysis
>> tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com
>>
> 

-- 
Derek M. Jones           Software analysis
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com