Much was written about two Artificial Intelligence systems developing their own language. Headlines like “Facebook shuts down down AI after it invents its own creepy language” and “Facebook engineers panic, pull plug on AI after bots develop their own language” were all over the place, seeming to imply that we were just at the verge of a significant incident in AI research.
As it happens, nothing significant really happened, and these headlines are only due to the inordinate appetite of the media for catastrophic news. Most AI systems currently under development have narrow application domains, and do not have the capabilities to develop their own general strategies, languages, or motivations.
To be fair, many AI systems do develop their own language. Whenever a neural network is trained to perform pattern recognition, for instance, a specific internal representation is chosen by the network to internally encode specific features of the pattern under analysis. When everything goes smoothly, these internal representations correspond to important concepts in the patterns under analysis (a wheel of car, say, or an eye) and are combined by the neural network to provide the output of interest. In fact, creating these internal representations, which, in a way, correspond to concepts in a language, is exactly one of the most interesting features of neural networks, and of deep neural networks, in particular.
Therefore, systems creating their own languages are nothing new, really. What happened with the Facebook agents that made the news was that two systems were being trained using a specific algorithm, a generative adversarial network. When this training method is used, two systems are trained against each other. The idea is that system A tries to make the task of system B more difficult and vice-versa. In this way, both systems evolve towards becoming better at their respective tasks, whatever they are. As this post clearly describes, the two systems were being trained at a specific negotiation task, and they communicated using English words. As the systems evolved, the systems started to use non-conventional combinations of words to exchange their information, leading to the seemingly strange language exchanges that led to the scary headlines, such as this one:
Bob: I can i i everything else
Alice: balls have zero to me to me to me to me to me to me to me to me to
Bob: you i everything else
Alice: balls have a ball to me to me to me to me to me to me to me to me
Strange as this exchange may look, nothing out of the ordinary was really happening. The neural network training algorithms were simply finding concept representations which were used by the agents to communicate their intentions in this specific negotiation task (which involved exchanging balls and other items).
The experience was stopped not because Facebook was afraid that some runaway explosive intelligence process was underway, but because the objective was to have the agents use plain English, and not a made up language.
Image: Picture taken at the Institute for Systems and Robotics of Técnico Lisboa, courtesy of IST.