[SystemSafety] Elephants, dinosaurs and integrating the VLA model

[Prof. Dr. Peter Bernard Ladkin]

On 2023-08-04 04:37 , Steve Tockey wrote:
>
> For what it’s worth, OWASP (Open Worldwide Application Security Project, www.owasp.org) released 
> their “OWASP Top 10 for LLM”. You can find the document here: 

Two things maybe to note.

First, when they (and a lot of other people, especially nowadays in "AI safety") write "safety" they 
are mainly talking about reliability, and I hope we here all still recognise that the two 
characteristica are different (both to system safety specialists and to the IEC).

Second, Bruce Schneier thinks that it won't "ever be possible to fully secure LLMs" against prompt 
injection attacks: see 
https://www.schneier.com/blog/archives/2023/07/automatically-finding-prompt-injection-attacks.html 
for his take on a piece of research which has shown how to automate such attacks.

Prompt injection is the latest incarnation of what has been the standard mode of network attack for 
some 30 years now. It started with buffer-overflow attacks on network service software in the 
1990's, (somewhat over 90% of the attacks registered by the then-new US-CERT) and has continued, 
with (according to my periodic informal scanning) 80-90% of the attacks on control gear registered 
by ICS-CERT being of the corrupted-input form.

I've often pointed out that we knew how to deal with this in the 1960's. It was called strong data 
typing and implemented in the personal computer's once-favorite programming language Pascal (and its 
grown-up version Modula). But the features of C that allow it to behave like assembly language won 
out in the "market" (a social phenomenon I still find puzzling - I can see why C was used for system 
programming, for which it was invented, but I don't understand why everyone else preferred it to 
Pascal). And we have been dealing with the consequences ever since, starting with buffer-overflow 
vulnerabilities. The paradigm likely needs some tweaking for current application, so I like to call 
it enhanced strong data typing (ESDT). I don't understand why it isn't an active area of computer 
scientific research, given that practical cybersecurity depends on it.

It worries me somewhat that Schneier thinks dependable ESDT is impossible. I don't see the 
reasoning. In particular, I don't see why you can't devise a reliable preprocessor for input 
(programmed using strong data typing) that parses incoming data for (a) being coherent natural 
language, and (b) not containing control sequences for the LLM software. (Neither of these two 
features need to be perfect - they can be overenthusiastic, for example, but it does need to be 
itself invulnerable to attack via specially-crafted input).

PBL

Prof. i.R. Dr. Peter Bernard Ladkin, Bielefeld, Germany
Tel+msg +49 (0)521 880 7319  www.rvs-bi.de