The Challenge of Debugging Artificial Intelligence

I am no expert in AI, but I have been very intimately involved in programming for most of my career in technology.  The explosive growth of AI is fascinating for me, because I have been hearing about AI forever (or at least since the mid-80’s).  For most of that time AI was mostly a very vague concept with a lot of hype and not much reality, unless one defines AI in the broadest terms as mimicking anything that human beings can do.  However, we do appear to be at a point now where AI as a true analog to human intelligence and learning will finally begin to impact our lives.

One interesting aspect of AI that distinguishes it from traditional computer programs is the fact that the program code does not specifically and logically specify what the program should do.  A non-AI computer program has a series of program steps with various data inputs that one can follow and see exactly what the program does.  If the behavior is different than expected, then one can trace the code and see why this occurs and correct it if desired.  This is called debugging.

 

AI does not have this kind of linear program flow that can be flowcharted or traced.  An AI program is intended to learn and evolve its behavior with new code and links based on the data it uses to learn and the learning mechanisms that have been built into its program.  This requires massive amounts of data and a massive amount of processing to create the new or evolved state of the program.  

 

As I understand it, once an AI program begins the process of learning it becomes essentially a black box. The relationships and connections it builds internally are too complex to be traced or followed in the way that one can do with non-AI programs.  If the output is undesirable or unexpected, then it seems likely that the only way to correct it is to have some sort of feedback mechanism that is meant to correct its behavior.  How quickly or efficiently that correction can be made is an interesting question.

 

This is analogous to human intelligence and behavior.  The brains of human beings are basically black boxes.  If a human being makes an error, there is no way to go in and trace the thinking or brain function and ‘debug’ the problem.  The only way to resolve bad or incorrect behavior is to give the human the appropriate feedback and hope that works.  

 

In many use cases, an AI robot or program will be unlikely to make errors that would be dangerous or problematic, but there are certainly cases where this could occur.  As in the human case, once an AI device is learning there is no definitive path that it will take.  It will be interesting to see how these devices perform over time and whether this unpredictability will become a major problem or not.

 

It is challenging enough to accommodate all the vagaries of human behavior and keep civilization on track. If we have robots and other AI devices with potentially unpredictable behavior it could push humanity right over the edge!