[SystemSafety] Explainability of "Algorithms"

[Chuck_Petras at selinc.com]

Just noticed this article, which is an author interview regarding her new 
mass market book, Weapons of Math Destruction.
https://urldefense.proofpoint.com/v2/url?u=http-3A__searchcio.techtarget.com_news_450421129_Mathematician-2Dwarns-2Dagainst-2Dweapons-2Dof-2Dmath-2Ddestruction&d=DwIBAg&c=zVFQZQ67ypsA9mYKSCqWmQHiVkCCaN-Gb60_N6TVnLk&r=zCwDz0h_ezUCVpbXoLT-zh0iTVdbymfdnT16kGAgelNE5W_nOFK-pESbjJCRy2gv&m=oSI_h2TGF9Tucd5fC_nPtk4oCRXPUT9sGsBQqYGJlKI&s=gCqW3K0boiDKfNyl4T0MYJsVb_TR3EPu8lw9CKhEZuk&e= 

"How do you define WMDs? 

"O'Neil: They're defined by their characteristics. There are three 
characteristics for WMDs: They're very important; they're widespread and 
are used on a lot of people for important decisions. They're secret; 
people don't understand how they're being scored, and sometimes they don't 
even understand that they're being scored."

"How do companies get to a point where they can be confident the 
algorithms they're using aren't biased to such a degree?

"O'Neil: For some reason, people separate out mathematical algorithms from 
other kinds of processes. So, in other words, if you were working in the 
unemployment office in Michigan and you were told, 'Hey, we have a new 
system that's going to be turned on tomorrow, and nobody understands it,' 
you would think everybody would keep a close eye on it to make sure it 
works. But for whatever reason, when it's algorithmic, people think, 'Oh, 
well, really smart people built this, so it's got to work.'
"So, I don't really have a very good explanation of it, except that people 
just trust mathematical algorithms too much. One of my major goals is a 
call for science and a warning away from blind trust. When I say science, 
what I mean is that I want evidence this works; I want evidence this is 
fair; I want evidence that the people who you're claiming are trying to 
defraud the system are actually trying to defraud the system, that you 
have a good accuracy rate, you have a low false positive rate, you have a 
low false negative rate. And, when you do have false positives, I want to 
make sure that they're not all falling on certain populations unfairly."

"Are machine learning algorithms -- or algorithms that teach themselves -- 
more or less likely to be weapons of math destruction?

"O'Neil: They're more likely because we don't really understand them. When 
you say they 'teach themselves,' it means we don't explicitly tell the 
algorithm how to interpret the data it's seeing. So, it interprets it in 
some kind of opaque way that we don't understand and can't explain.
"That's already a bad sign. And the other thing that's bad is that people 
trust deep learning, neural network stuff even more than they trust other 
kinds of statistical methods. It's, again, the blind trust problem."

In the second part of this two-part Q&A, O'Neil suggests it might be time 
for a national safety board for algorithms. 
https://urldefense.proofpoint.com/v2/url?u=http-3A__searchcio.techtarget.com_news_450421130_Data-2Dskeptic-2DCathy-2DONeil-2Dexplains-2Dwhy-2Dwe-2Dneed-2Dto-2Dregulate-2Dalgorithms&d=DwIBAg&c=zVFQZQ67ypsA9mYKSCqWmQHiVkCCaN-Gb60_N6TVnLk&r=zCwDz0h_ezUCVpbXoLT-zh0iTVdbymfdnT16kGAgelNE5W_nOFK-pESbjJCRy2gv&m=oSI_h2TGF9Tucd5fC_nPtk4oCRXPUT9sGsBQqYGJlKI&s=OGlQDE3nKkWenqximnMYPak2BijgbMqAu6Ed1uJVy24&e= 


Chuck Petras, PE**
Schweitzer Engineering Laboratories, Inc
Pullman, WA  99163  USA
http://www.selinc.com
Tel: +1.509.332.1890
Cel: +1.509.991.0613

SEL Synchrophasors - A New View of the Power System <http://synchrophasor.selinc.com>

Making Electric Power Safer, More Reliable, and More Economical (R)

** Registered in Oregon.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20170628/8f11d4fd/attachment.html>