Facebook Shuts Down Negotiating AI After It Created Its Own Language

Back in June, we ran a story on how Facebook had created chat bots capable of negotiating and, inadvertently, also taught them how to lie. The bots lied by learning from real human conversations and subsequently faking interest in one item to add perceived value. Now, these Artificial Intelligence (AI) agents have been shut down for creating a language of their own in order to communicate more efficiently.

The Bots That Gave up on English

In a negotiating experiment, Facebook’s AI bots were shown two books, one hat, and three balls. Each agent was assigned a different value that represented how much it cared about a specific item, and the goal was to maximize profit by negotiating with the other bots.

Now, according to Fast Co. Design, Facebook’s negotiating AI apparently decided that the English language was not efficient enough for its negotiations and decided to create its own language to get rid of its inconsistencies.

The resulting language looks like the gibberish one would expect out of AI built by amateurs, but upon further inspection, it was decided that despite being seemingly unintelligible to humans, it made perfect sense to AI agents. Indeed, it allowed them to communicate with one another without researchers knowing what was going on.

An exchange between two AI agents, Bob and Alice, went like this: Bob stated, “I can i i everything else”, and Alice came back at him with, “balls have zero to me to me to me to me…” The conversation continued from there with variations of these two sentences.

According to researchers, repeating phrases like “i” and “to me” reflect how AI operates. The two bots were working out how many of each item each one should take. Bob’s words “i i everything else” showed that he was offering Alice more items. Apparently, English use was not rewarding for AIs in this scenario, and they utilized a more efficient solution given the way in which they operate.

Per Facebook AI researcher Dhruv Batra:

“Agents will drift off from understandable language and invent code-words for themselves. Like if I say ‘the’ five times, you interpret that to mean I want five copies of this item. This isn’t so different from the way communities of humans create shorthands.”

Facebook Will Stick to English-Speaking AI

Researchers noted that as there are no humans capable of understanding this type of language, they’ve had to shut these agents down and stick to those that use English so they can stay in charge of what is going on.

After all, the goal of teaching bots to chat and negotiate is getting them to talk to humans, not to one another. If we are looking for the best restaurant in town, gibberish is not going to help us out.

In a similar case, Google’s AI managed to improve its translation service by learning language pairs it hadn’t been taught, seemingly by using its own internal language. For example, if an agent had been taught English to French, and French to Italian, it could translate English to Italian.