[SystemSafety] AI and safety

[paul_e.bennett at topmail.co.uk]

On 12/11/2018 at 7:59 AM, "Dariusz Walter" <dariusz at walterki.com> wrote:
>
>Based on the brief descriptions in the database, all the AI 
>solutions seem
>to play within the set rules that are explicitly defined. None of 
>the AI
>solutions seemed to have "failed" any of the set rules. They 
>definitely
>seem to be reliable. In fact, on review of the issues in the 
>database, I
>find the AI solutions ingenious.
>
>If anything, in my mind they identify the holes/gaps/assumptions 
>present in
>the explicit set of rules defined for the task, including
>
>1. the specifications/behaviour of the environments that these AI 
>systems
>are grown to work in
>
>2. completeness/correctness of the rules/cost functions that these 
>AI
>systems are supposed to meet
>
>Imagine if some of the cost criteria/requirements that these AI 
>systems are
>grown for were safety requirements. It would be interesting to see 
>what
>ingenious solutions would be identified, and if they would in fact 
>be safe
>to a human interpretation.

Based on the above, one could almost consider that using AI in the
threat development landscape as a means to test out safety systems.
In regards of generating a wide range of risk scenarios that could
breach security and safety protocols, AI might become one basis
to provide the harshest of tests.

Just a thought.

I still think that safety systems, that are as simple as possible, kept in
a zone that is as secure as can be made, in situations that receive
adequate and ongoing review throughout development and operation,
is the way we do our best job. The standards are good guidance for
how to get there, but that does not mean we cannot innovate to make
the best systems.

Regards

Paul E. Bennett IEng MIET
Systems Engineer
Lunar Mission One Ambassador
-- 
********************************************************************
Paul E. Bennett IEng MIET.....
Forth based HIDECS Consultancy.............
Mob: +44 (0)7811-639972
Tel: Due to relocation - new number TBA. Please use Mobile.
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************