[SystemSafety] MC/DC coverage assumptions

[Dewi Daniels]

Derek,

I don't think that's what Ian is saying at all. There are structural
coverage tools that measure MC/DC as an optional extra step. Some of those
tools only consider decisions with multiple conditions in their separate
MC/DC analysis, as decisions with single conditions will have already been
addressed by the decision coverage analysis, so there's nothing more to
analyse. As Ian wrote, it would be wrong to claim MC/DC for the single
condition decisions just because they weren't included in the separate
MC/DC analysis. You'd also have to look at the output of the decision
coverage analysis. It sounds as if the authors of the paper didn't know how
to drive the tool they were using.

Yours,

Dewi Daniels | Director | Software Safety Limited

Telephone +44 7968 837742 | Email d <ddaniels at verocel.com>
ewi.daniels at software-safety.com

Software Safety Limited is a company registered in England and Wales.
Company number: 9390590. Registered office: Fairfield, 30F Bratton Road,
West Ashton, Trowbridge, United Kingdom BA14 6AZ

On 28 February 2018 at 15:56, Derek M Jones <derek at knosof.co.uk> wrote:

> Dewi,
>
> The definition of Modified Condition/Decision Coverage is:
>>
> ...
>
>> In your example, the single condition would have to have been evaluated to
>> both true and false to achieve MC/DC. This means that at least two test
>> cases are required to achieve either MC/DC or branch coverage, but one
>> test
>> case would be sufficient to achieve statement coverage.
>>
>
> As Ian Broster's interesting reply shows, there are people out there who
> have other ideas.
>
> I'd be astonished if a tool with the pedigree of LDRA Testbed got this
>> wrong.
>>
>
> As would I.
>
> Perhaps there is a compatibility flag, -do_what_broken_tool_does?
>
>
>> Dewi
>>
>> Yours,
>>
>> Dewi Daniels | Director | Software Safety Limited
>>
>> Telephone +44 7968 837742 | Email d <ddaniels at verocel.com>
>>
>> ewi.daniels at software-safety.com
>>
>> Software Safety Limited is a company registered in England and Wales.
>> Company number: 9390590. Registered office: Fairfield, 30F Bratton Road,
>> West Ashton, Trowbridge, United Kingdom BA14 6AZ
>>
>> On 28 February 2018 at 14:35, Derek M Jones <derek at knosof.co.uk> wrote:
>>
>> All,
>>>
>>> I was recently reading a paper that compared unit testing of
>>> industrial embedded software with some open source programs.
>>> The comparison included a table of statement, branch and MC/DC coverage,
>>> items in the table included: aerospace software, automotive software and
>>> subway signal software
>>>
>>> The MC/DC coverage numbers were a lot better than the statement and
>>> branch coverage.  This is obviously a mistake, at best they can be
>>> as good as.
>>>
>>> I emailed the authors, who have been very prompt replying.
>>> The latest reply was a bit surprising.
>>>
>>> The algorithm they used for MC/DC assumes that a function containing
>>> a single branch (e.g., an if-statement with no else part) and
>>> the test involves a single condition (i.e., no AND or OR conditions),
>>> then 100% MC/DC coverage is assumed, even if 100% branch coverage is
>>> not obtained.
>>>
>>> Sounds like a mistake in their algorithm.  However, they claim there is
>>> some amount of existing practice and even call out Testbed as
>>> behaving like this (I don't have a copy to check this out).
>>>
>>> Somebody please tell me that this is not an assumption made by
>>> commercial packages when calculating MC/DC coverage.
>>>
>>> The authors admit that MC/DC coverage cannot be better than
>>> statement and branch coverage, and admit the current presentation
>>> of MC/DC coverage in the table could be misleading.  They are going
>>> to release a version with corrected data.
>>>
>>> --
>>> Derek M. Jones           Software analysis
>>> tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com
>>> _______________________________________________
>>> The System Safety Mailing List
>>> systemsafety at TechFak.Uni-Bielefeld.DE
>>>
>>>
>>
> --
> Derek M. Jones           Software analysis
> tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180228/a44c457e/attachment.html>