[SystemSafety] MC/DC coverage assumptions

[Steve Tockey]

Alex wrote:

³p.s.: I don't really know what MC/DC means if the conditions are subject
to change. I haven't seen a tool that is
able to handle this...

if (a
#if FOO
    || b
#endif
   )
   /* something */²

I am certainly not the expert, but it would seem likely that since ³#if
FOO² is only relevant at compile-time, not at run-time, then both compiled
versions should warrant different testing. The MC/DC test cases for the
version compiled with FOO == false can safely ignore b. This would be
trivial since "if(a)² is automatically MC/DC with only two test cases: a
== true and a == false. On the other hand, the MC/DC test cases for the
version compiled with FOO == true cannot ignore b. To get MC/DC, three
test cases are needed:

Test case 1) a == false, b == false: expected result is that something is
not done
Test case 2) a == true, b == false: expected result is that something is
done
Test case 3) a == false, b == true: expected result is that something is
done

Test case 1 pairs with Test case 2 to show the independence of condition
a. Test case 1 pairs with Test case 3 to show the independence of
condition b.


That said, if one were to run Test cases 1, 2, and 3 against the version
compiled with FOO == false, Test cases 1 and 2 alone would be sufficient
for MC/DC. Test case 3 is problematic in that it expects that something
will be done yet something will not be done because a == false and b is
irrelevant. So it can¹t be just a matter of throwing the MC/DC test cases
for the version compiled when FOO == true against the version compiled
when FOO == false because Test case 3 gives a false failure. The MC/DC
test cases need to be sensitive to the value of FOO as well. Specifically,
in an automated test environment:


Test case 1) a == false, b == false: expected result is that something is
not done
Test case 2) a == true, b == false: expected result is that something is
done
#if FOO
Test case 3) a == false, b == true: expected result is that something is
done
#endif



Cheers,

‹ steve



-----Original Message-----
From: systemsafety <systemsafety-bounces at lists.techfak.uni-bielefeld.de>
on behalf of "Alexander.Much at elektrobit.com"
<Alexander.Much at elektrobit.com>
Date: Wednesday, February 28, 2018 at 12:05 PM
To: "derek at knosof.co.uk" <derek at knosof.co.uk>,
"systemsafety at lists.techfak.uni-bielefeld.de"
<systemsafety at lists.techfak.uni-bielefeld.de>
Subject: Re: [SystemSafety] MC/DC coverage assumptions

Hi Derek, *,

[...]
> The authors admit that MC/DC coverage cannot be better than statement
> and branch coverage, and admit the current presentation of MC/DC
> coverage in the table could be misleading.  They are going to release a
> version with corrected data.
> 

void f (void)
{
#if A || B
   something
#endif

   if (a || b)
   {
     /* something */
   }
}

Most tools don't consider the branches or conditions in the pre-processor:
they only see one variant and instrument it.

That's why we include statement coverage instrumentation in our testing
*in addition* to tool-based MC/DC.

Just 2c,
Alex

p.s.: I don't really know what MC/DC means if the conditions are subject
to change. I haven't seen a tool that is
able to handle this...

if (a
#if FOO
    || b
#endif
   )
   /* something */

--
Alexander Much
Chief Expert - Head of Software Systems Engineering

EB - Driving the Future of Software
P +49 9131 7701 6384
M +49 172 7479804
E alexander.much at elektrobit.com

Elektrobit Automotive GmbH, Am Wolfsmantel 46, 91058 Erlangen, Germany
Managing Directors: Alexander Kocher, Gregor Zink; Register Court Fürth
HRB 4886



_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE