[SystemSafety] MC/DC coverage assumptions

[Steve Tockey]

Derek,
Ok, thanks for the explanation. That makes sense. Yes, if you have
achieved 100% MC/DC then you must also have achieved 100% Decision
coverage (but not the other way around). And if you have achieved 100%
Decision coverage then you must also have achieved 100% Statement coverage
(but, again, not the other way).


Cheers,

― steve



-----Original Message-----
From: systemsafety <systemsafety-bounces at lists.techfak.uni-bielefeld.de>
on behalf of Derek M Jones <derek at knosof.co.uk>
Organization: Knowledge Software, Ltd
Date: Wednesday, February 28, 2018 at 8:10 AM
To: "systemsafety at lists.techfak.uni-bielefeld.de"
<systemsafety at lists.techfak.uni-bielefeld.de>
Subject: Re: [SystemSafety] MC/DC coverage assumptions

Steve,

> ³The MC/DC coverage numbers were a lot better than the statement and
> branch coverage.  This is obviously a mistake, at best they can be
> as good as.²
> 
> What does ³coverage number² mean here? More defects revealed? More test
> cases required?

The number of functions having a given percentage coverage.
The paper had almost all functions with MC/DC at 100% (which
implies statement coverage for those functions is 100%, which
it was not).


> -----Original Message-----
> From: systemsafety <systemsafety-bounces at lists.techfak.uni-bielefeld.de>
> on behalf of Derek M Jones <derek at knosof.co.uk>
> Organization: Knowledge Software, Ltd
> Date: Wednesday, February 28, 2018 at 6:35 AM
> To: "systemsafety at lists.techfak.uni-bielefeld.de"
> <systemsafety at lists.techfak.uni-bielefeld.de>
> Subject: [SystemSafety] MC/DC coverage assumptions
> 
> All,
> 
> I was recently reading a paper that compared unit testing of
> industrial embedded software with some open source programs.
> The comparison included a table of statement, branch and MC/DC coverage,
> items in the table included: aerospace software, automotive software and
> subway signal software
> 
> The MC/DC coverage numbers were a lot better than the statement and
> branch coverage.  This is obviously a mistake, at best they can be
> as good as.
> 
> I emailed the authors, who have been very prompt replying.
> The latest reply was a bit surprising.
> 
> The algorithm they used for MC/DC assumes that a function containing
> a single branch (e.g., an if-statement with no else part) and
> the test involves a single condition (i.e., no AND or OR conditions),
> then 100% MC/DC coverage is assumed, even if 100% branch coverage is
> not obtained.
> 
> Sounds like a mistake in their algorithm.  However, they claim there is
> some amount of existing practice and even call out Testbed as
> behaving like this (I don't have a copy to check this out).
> 
> Somebody please tell me that this is not an assumption made by
> commercial packages when calculating MC/DC coverage.
> 
> The authors admit that MC/DC coverage cannot be better than
> statement and branch coverage, and admit the current presentation
> of MC/DC coverage in the table could be misleading.  They are going
> to release a version with corrected data.
> 

-- 
Derek M. Jones           Software analysis
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com
_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE