[SystemSafety] Accuracy of COVID modeling code

[Derek M Jones]

Thierry,

> Personally, I would not have used the SW technologies they used for reliable scientific computations, with the increased risk of software defects, but that is probably not a consideration retained by that team, for a quick prototype and a work in progress.


I'm not aware of any evidence showing that any language is any better
than any other language with regard to reliable scientific computations.

Please let me know if you are aware of any such published evidence.

The claims of one often cited paper was been well and truely shot to pieces:
http://shape-of-code.coding-guidelines.com/2019/11/20/a-study-a-replication-and-a-rebuttal-se-research-is-starting-to-become-serious/

The code for the original Imperial model was more likely to be correct
because it was short and semantically close to the original maths (the correctness
of the maths is another issue).  Such an implementation was possible
because they used STAN, which happened to supports exactly what they wanted,
so you then have the issue of the correctness of the substantial STAN code.

This latest model looks completely home grown, probably because they had
no other option.


> Best regards,
> 
> Thierry Coq
> DNV GL FRANCE SARL
> PS. This is my personal response and does not engage my employer.
> 
> 
> 
> -----Original Message-----
> From: systemsafety <systemsafety-bounces at lists.techfak.uni-bielefeld.de> On Behalf Of Derek M Jones
> Sent: mercredi 6 mai 2020 14:02
> To: systemsafety at techfak.uni-bielefeld.de
> Subject: [SystemSafety] Accuracy of COVID modeling code
> 
> All,
> 
> Governments are making Coronavirus decisions based on the predictions of models.
> 
> Model predictions depend on the accuracy of the model and the accuracy of the code implementing the model.
> 
> Some comments on the original, broad brush model:
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fshape-of-code.coding-guidelines.com%2F2020%2F04%2F05%2Fcomments-on-the-covid-19-model-source-code-from-imperial%2F&data=02%7C01%7Cthierry.coq%40dnvgl.com%7C73b24b9ac2524dc83f4b08d7f1b55721%7Cadf10e2bb6e941d6be2fc12bb566019c%7C1%7C0%7C637243633462475102&sdata=eQsPgbV4DbC8VCmuhT4dXORyHQTJOqqaln9qbuBslC0%3D&reserved=0
> 
> Now we have a detailed model, which I suspect is very new:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmrc-ide%2Fcovid-sim&data=02%7C01%7Cthierry.coq%40dnvgl.com%7C73b24b9ac2524dc83f4b08d7f1b55721%7Cadf10e2bb6e941d6be2fc12bb566019c%7C1%7C0%7C637243633462475102&sdata=AeDpiOSV5iTFodYAjScKL4CSlHDY1HgZc6rKqr9ouls%3D&reserved=0
> 
> I have not had much luck in drumming up reviewers for the earlier models.
> 
> Please reach out to people with the expertise to review this new code.
> 
> --
> Derek M. Jones           Evidence-based software engineering
> tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> Manage your subscription: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.techfak.uni-bielefeld.de%2Fmailman%2Flistinfo%2Fsystemsafety&data=02%7C01%7Cthierry.coq%40dnvgl.com%7C73b24b9ac2524dc83f4b08d7f1b55721%7Cadf10e2bb6e941d6be2fc12bb566019c%7C1%7C0%7C637243633462475102&sdata=1tNxOwxoAJ8h1m3xOisYon3tMMgBHArrukl0LF9ZztQ%3D&reserved=0
> 
> **************************************************************************************
> This e-mail and any attachments thereto may contain confidential information and/or information protected by intellectual property rights for the exclusive attention of the intended addressees named above. If you have received this transmission in error, please immediately notify the sender by return e-mail and delete this message and its attachments. Unauthorized use, copying or further full or partial distribution of this e-mail or its contents is prohibited.
> **************************************************************************************
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety
> 

-- 
Derek M. Jones           Evidence-based software engineering
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com