[SystemSafety] Koopman replies to concerns over Toyota UA case

Mario Gleirscher mario.gleirscher at tum.de
Sat Jan 6 12:17:30 CET 2018


Dear Steve,
happy new year to everyone,

On 30.12.2017 22:03, Steve Tockey wrote:
> A whole bunch of
> vitally important research needs to be done here.

I am following this thread with high excitement and just want to let you
know about work that a few colleagues and I were able to publish a few
years back.

https://rd.springer.com/article/10.1007/s11219-013-9217-z

My conclusion from applying several static analysis tools to several
medium size SW systems (not really safety critical SW but in practical
use), beyond the article, was actually quite disappointing, of course we
tried to reduce bias, opinion, and subjectivity as far as possible:

The largest fraction of measurements (i.e. rule applications) was
literally meaningless, a small fraction of measurements is kind of
interesting but not really critical. And only very few out of hundreds
and thousands of measurements (i.e. <<1%) were somehow
critical, depending on the system/context of course (unfortunately we
did not try to subsume rules into the MISRA framework). The final
criticality rating was highly depending on a review of the location of
the finding in the code and differed quite often from what the tool
reports suggested. Sure, a proper setup and adjustment of these tools is
necessary, however, demanding, takes a lot of time, and is different for
every other system. Yet in the study, the overall perception was more
positive than negative, at least we have tools that point to some things ...

What I took away from being involved in this insightful project was that
far fewer things are correlated than expected, and the majority of the
rules seem to be far away from being usable as _critical_ defect
predictors and this is what counts imho.  Moreover, it is necessary to
state that only few of the rules in the tools we used cover relative
correctness, an objective of "formal methods"- or "semantics"-based
tools. For sure, some of the rules help cleaning up "shaghetti code" and
deliver eye-friendlier, more easily maintainable source.

After all I can still encourage empirical SW researchers to confirm or
rebut my feelings about some these static code checking rules.

I also suggest to separate what we can and want to do for code
maintainability and what we can and have to for dependability,
reliability, security, and safety. I guess, classical static analysis is
a very small piece in the cake of getting SW there where we want it to be.

Best wishes for a successful new year,
Mario


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5053 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180106/b37a9f09/attachment.bin>


More information about the systemsafety mailing list