Facebook, or as we’re supposed to call them now Meta, announced earlier today that its artificial intelligence CICERO has achieved “human-level performance” in the board game. Diplomacywhich is distinguished by the fact that it is a game based on human interaction, and not on movements and maneuvers (like, for example, chess).
Here’s a frankly agonizing trailer:
If you have never played Diplomacy, and might be wondering what the problem is, it’s a board game first released in the 1950s that’s played mostly by people sitting around a table (or splitting into rooms) and trading stuff. There are no dice or cards affecting the game; everything is determined by humans communicating with other humans.
So for the creators of an AI to say that it plays at a “human level” in a game like this is a pretty bold claim! One which Meta supports by saying that CICERO actually operates on two different levels, one analyzing game progress and state, the other trying to communicate with the human levels in a way that we would understand and with which we would interact.
Meta appealed to “world champion diplomacy” Andrew Goff to back up their claims, who says “a lot of human players will soften their approach or they’ll start being motivated by revenge and CICERO never does that. He just plays the situation as he sees it, so he is ruthless in executing his strategy, but he is not ruthless in a way that annoys or frustrates other players.
Sounds optimal, but as Goff says, maybe too optimal. Which reflects the fact that while CICERO plays well enough to keep up with humans, he’s far from perfect. As Meta themselves say in a blog post, CICERO “sometimes generates incoherent dialogue that can undermine his goals”, and my own criticism would be that every example they provide of his communication (like the one below) makes him look like a terrified psychopathic office worker that he’s they don’t end every sentence with!!! you will think they are a terrible person.
Of course, the ultimate goal of this program is not to win board games. It is simply by using Diplomacy as a “sandbox” to “advance human-AI interaction”:
Although CICERO is only capable of playing Diplomacy, the technology behind this achievement is relevant to many real-world applications. Controlling natural language generation through scheduling and RL could, for example, reduce communication barriers between humans and AI-powered agents. For example, today’s AI assistants excel at simple question-and-answer tasks, like telling you the weather, but what if they could sustain a long-term conversation with the goal of teaching you a new skill? Alternatively, imagine a video game where non-player characters (NPCs) could plan and converse like people – understanding your motivations and tailoring the conversation accordingly – to aid you in your quest to storm the castle.
I may not be a billionaire Facebook executive, but instead of spending all this time and money improving AI assistants, which no one outside of AI research and corporate spending seems to care, couldn’t we just… hire some humans I can talk to instead?