[SystemSafety] Perturbing internal state may not change external behavior

[Derek M Jones]

All,

Those who have dived into the details of large language models
know that their implementation mostly consists of lots of matrix
adds and multiplies
https://theaisummer.com/transformer/#the-core-building-block-multi-head-attention-and-parallel-implementation

If you do enough multiplications, some unexpected behavior
frequently occurs
https://shape-of-code.com/2023/06/11/perturbed-expressions-may-recover/

Large numbers of multiplications also crop up a lot in genetically
generated code.

The fact that perturbations to internal state can die out
is not unique to matrix multiplication
https://shape-of-code.com/2023/04/02/percolation-of-the-impact-of-coding-mistakes-through-a-program/

-- 
Derek M. Jones           Evidence-based software engineering
blog:https://shape-of-code.com