[SystemSafety] Comparing reliability predictions with reality

[Derek M Jones]

All,

Having spent some time reading lots of papers on
models of software reliability, based on
cpu time between fault experiences, I have not found
one that measures the accuracy of the models.  By
accuracy I mean comparison of the prediction of time
to next fault against actual time to next fault.

There is something of a cottage industry of papers
that compare probability distributions fitted to historical
data.  Researchers seem shy about taking the next step of
comparing predictions of future faults.  Perhaps because
the results are not very good?

Does anybody know of papers that compare predictions against
actual?

Yes, lack of data is a perennial problem.

Some related discussion here
https://shape-of-code.com/2025/02/23/deep-dive-looking-for-good-enough-reliability-models/

-- 
Derek M. Jones           Evidence-based software engineering
blog:https://shape-of-code.com