[SystemSafety] AI Hallucination Cases

[Les Chambers]

Guys
I’m excited to see this list turning to discussion of AI risks.
Full disclosure – I am a regular user (minute to minute) of Gemini Deep 
Research, Perplexity, Claude and ChatGPT XX. My problem domains range from how 
to put my rented late model Citroen into reverse - to the best Paris Metro/RER 
connections to get from the Marais District to Monet’s Giverny house and 
gardens - to a layman-friendly parsing of a 500 word MRI scan report (Gemini 
DR indicates no worrysome anomalies). 
IMHO hallucinations in these domains are not a huge problem. You can judge 
authenticity by triangulation - posing the same prompt to at least 3 AIs or 
engaging your brain with critical thinking. In the limit you could always ask 
a human being as I plan to do in an appointment with my oncologist 
momentarily. 
The hallucination issue, if not the horror, as I see it is the LLM in the self 
driving motor vehicle coming for you in the opposite direction at a relative 
speed of 200 km/h +. I recently drove 200 km on Scotland’s single track roads. 
I was charmed by both the scenery and the courtesy of the oncoming drivers. 
Turn-outs abounded and oncoming drivers stopped to give you time to reach 
yours. My question is, “What is the probability that the robots and will 
extend us the same social grace.”
I could go on but before I do I’d recommend viewing an insightful and well 
informed presentation by Andrej Karpathy who spent five years at the heart of 
Tesla's autopilot development.
https://bit.ly/44zwp8d
Paraphrasing his statement that drew my attention (read RED FLAG):
“At Tesla the neural networks progressively took over functions of the code. 
For example stitching the output of the various cameras into a single image. 
The neural network ate the code.”
Ergo, the proposition I believe this list should be debating is as follows:
Given the inexorable replacement of procedural code by LLMs in safety critical 
systems, how can safety critical systems engineers hope to guarantee the 
general public protection from harm when the life-critical systems frameworks 
and procedures we have just lovingly spent 50 years developing are 
progressively becoming irrelevant - given they are based on the deployment of 
procedural code.

Over to you 


Cheers 
Les
> 
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
> Manage your subscription: https://lists.techfak.uni-
bielefeld.de/mailman/listinfo/systemsafety



--
Les Chambers
les at chambers.com.au

https://www.chambers.com.au
https://www.systemsengineeringblog.com

+61 (0)412 648 992