[SystemSafety] AI and the virtuous test Oracle

[Les Chambers]

Hi All

I find myself reflecting on what will become of us. 
As systems engineering best practice is overrun by AI.

Practitioners report that neural networks are eating code. 
Example 1: The vector field surrounding a Tesla motor vehicle is an output of a 
neural network, not the result of software logic. Soon the neural net - not 
code - will 
generate controls. The size of the code base is reducing.  (Elon 
Musk)
Example 2: the ChatGPT transformer code base is only 2000 LOC (Mo Gawdat 
https://youtu.be/bk-nQ7HF6k4)

The intelligence resides in terabytes of data, perceptrons and millions of 
weighting parameters. All are gathered by automated means. Not subject to human 
review.

Ergo what will become of our trusty barriers to dangerous failure:
1. Safety functions - gone
2. Verification - gone
3. Code reviews - gone
4. Validation - How?

On validation, may I suggest the moral AI. A test oracle built on a virtuous 
dataset, capable of interrogating the target system to determine virtue. Test 
outcomes will morph from pass/failure to moral/immoral. 

Credible industry players have predicted that soon we will have AIs orders of 
magnitude smarter than us. Especially when they start talking to each other. 
The bandwidth will be eye-watering - the increase in intelligence, vertical.

New barriers are required. Time to develop an AI that is on our side – the side 
of ethics and the moral life. An adult in the room if you like. We should birth 
this creature now and raise it as good parents.

Let us not panic. May I put the proposition: virtue, like creativity, can be 
algorithmic.
I have a sense of starting from the beginning - tabula rasa. I suggest that 
high-level thinking on the subject could begin with ChatGPT prompts: 
1. What is the stoic philosopher’s concept of virtue?
2. What are the elements of philosophy relevant to AI?

Let us not forget our engineering mission: Guardians of the divine Logos, the 
organizing principle of the universe, responsible for its creation, 
maintenance, and order.

Would anyone care to riff on this?

Les

--

Les Chambers

les at chambers.com.au
systemsengineeringblog.com

+61 (0)412 648 992