[SystemSafety] AI Hallucination Cases

[Derek M Jones]

Paul,

> The latter book points out that we have all been hoodwinked into anthropomorphising LLMs, e.g. using the word 
> "hallucinate" which creates the impression there's a mind in the matrices.

Whenever people use terms like 'understand' when talking about LLMs,
I point out that it's next token prediction, which often annoys them.

When LLMs first arrived I had trouble believing it was all next
token prediction.  I read the "Attention is all you need" paper
and understood enough to see it was predicting the next token,
so I checked the code, and yes, next token prediction.
If you think that companies have some secret sauce, you can download
an open weights model and run it locally (yes, it's slow)
https://lmstudio.ai/

It's also possible to build an LLM that predicts the previous token,
i.e., it generates text right to left, rather than left to right
https://arxiv.org/html/2410.09817v1
However, the performance is not as good.

"I don't know" is a token sequence that rarely has a non-zero
occurrence probability in the training data.

One of the early LLMs said that the author of a book I wrote
had a degree in Divinity studies, which I enjoyed having for
a month or two.

  They recommend talking about  "stochastic
> parrot" or "mathy maths" instead of LLMs, and note that the entire purpose of an LLM is just to "make s**t up" on the 
> basis of weighted averages from training data.
> 
> br
> Paul
> 
> [1] https://www.amazon.co.uk/More-Everything-Forever-Overlords-Humanity/dp/B0F2N3F339
> [2] https://www.amazon.co.uk/AI-Fight-Techs-Create-Future/dp/B0DQQD5XML
> 
> On 2025-07-16 14:28, Derek M Jones wrote:
>> All,
>>
>> This database tracks legal decisions in cases where generative AI
>> produced hallucinated content – typically fake citations, but
>> also other types of arguments.
>>
>> 212 cases and counting
>>
>> https://www.damiencharlotin.com/hallucinations/
>>
>> via https://www.data-is-plural.com

-- 
Derek M. Jones           Evidence-based software engineering
blog:https://shape-of-code.com