[SystemSafety] Faults dependent on system state

[Steve Tockey]

Derek,
It turns out to be trivial to create scenarios where defects are not guaranteed to be revealed through control-flow coverage testing alone. Data-flow testing (specifically, All Uses and All DU Paths coverage), while not at all popular in the software industry, can reveal defects that might not be revealed by even MC/DC coverage. So I’m not at all surprised. Without having fully read the referenced article, the question might be, “would data-flow coverage testing have revealed them?” I’ll need to look into that when I have some more breathing room.


Cheers,

— steve



On Jul 11, 2023, at 5:25 PM, Derek M Jones <derek at knosof.co.uk> wrote:

All,

The SQLite database is tested with 1.7 million tests
that provides 100% branch test coverage and
100% MC/DC test coverage.  And yet still contains
coding mistakes which generate fault experiences.

The authors of this paper show how some faults will
only be experienced in a given system state (which
databases have lots of).

https://www.usenix.org/system/files/osdi20-rigger.pdf


-- 
Derek M. Jones           Evidence-based software engineering
blog:https://shape-of-code.com
_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE
Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety