Facebook Shuts Down Negotiating AI After It Created Its Own Language

Back in June, we ran a story on how Facebook had created chat bots capable of negotiating and, inadvertently, also taught them how to lie. The bots lied by learning from real human conversations and subsequently faking interest in one item to add perceived value. Now, these Artificial Intelligence (AI) agents have been shut down for creating a language of their own in order to communicate more efficiently.

The Bots That Gave up on English

In a negotiating experiment, Facebook’s AI bots were shown two books, one hat, and three balls. Each agent was assigned a different value that represented how much it cared about a specific item, and the goal was to maximize profit by negotiating with the other bots.

Now, according to Fast Co. Design, Facebook’s negotiating AI apparently decided that the English language was not efficient enough for its negotiations and decided to create its own language to get rid of its inconsistencies.

The resulting language looks like the gibberish one would expect out of AI built by amateurs, but upon further inspection, it was decided that despite being seemingly unintelligible to humans, it made perfect sense to AI agents. Indeed, it allowed them to communicate with one another without researchers knowing what was going on.

An exchange between two AI agents, Bob and Alice, went like this: Bob stated, “I can i i everything else”, and Alice came back at him with, “balls have zero to me to me to me to me…” The conversation continued from there with variations of these two sentences.

According to researchers, repeating phrases like “i” and “to me” reflect how AI operates. The two bots were working out how many of each item each one should take. Bob’s words “i i everything else” showed that he was offering Alice more items. Apparently, English use was not rewarding for AIs in this scenario, and they utilized a more efficient solution given the way in which they operate.

Per Facebook AI researcher Dhruv Batra:

“Agents will drift off from understandable language and invent code-words for themselves. Like if I say ‘the’ five times, you interpret that to mean I want five copies of this item. This isn’t so different from the way communities of humans create shorthands.”

Facebook Will Stick to English-Speaking AI

Researchers noted that as there are no humans capable of understanding this type of language, they’ve had to shut these agents down and stick to those that use English so they can stay in charge of what is going on.

After all, the goal of teaching bots to chat and negotiate is getting them to talk to humans, not to one another. If we are looking for the best restaurant in town, gibberish is not going to help us out.

In a similar case, Google’s AI managed to improve its translation service by learning language pairs it hadn’t been taught, seemingly by using its own internal language. For example, if an agent had been taught English to French, and French to Italian, it could translate English to Italian.

  • Ravi Bhojwani

    LInked In and Skype also at Risk..

  • Mats Ahlqvist

    So although bots are Artificiall Intelligence they are not able to enrich or change language only interpret and use for communication. Perhaps this is best for the english language but I think other languages could use some artificial intelligent suggestions as how to use the language in a better more adequate way.

  • fattypanda1

    The start of SKYNET … we humans will be paid in cryptos by our new master … ROBOTS or super AIs…. for our labour as their slaves ..if we are lucky. The not so lucky will be canned as luncheon meat to feed livestocks.

  • Steinar Kolnes

    Indeed very interesting!
    In Norway during WWII and the nazi occupation, the local Norwegians, especially here on the west coast developed what they called the “crow language”. By removing redundancy in the language they made it totally foreign for the nazis and even for Norwegians who did not know the “coding”. I Norway the nazis executed peoples just telling news so it was dead serious business since all readios were confiscated, My grandfather told me that the crow language was so effective (you could say more in shorter time), that they even used it after WWI. But it died out after a while. Of course it needed some real time processing in the human brain. But still there are old peoples including my aunty Berta who understand and can still speak it. So back to AI, it is not a surprise that the computers developed the same type of secret, local language 🙂