Long Talk Short

Thoughts, stories and ideas.

A Parlor-Game Leaderboard

Dwarkesh Patel ranked AIs at Blood on the Clocktower. GPT-6 Astro leads after 368 games — inside one parlor game, not as a general lie detector.

Read aloud in your browser

A parlor-game Elo is not a verdict on AI honesty in the wild. Dwarkesh Patel spent about $20,000 in API credits ranking models at Blood on the Clocktower — a social-deception game — and reports GPT-6 Astro ahead of Claude Opus 4.6 after 368 whole-team matches. The useful question is what skill the harness actually measures: long-horizon bluffing and detection inside one ruleset, not general deception.

In short: Patel ran 368 games in which one model filled every seat on the good team and another filled every seat on the evil team. In his telling, OpenAI's GPT-6 Astro leads, with a large Elo gap over Anthropic's Claude Opus 4.6. The setup is clever and the play-by-play is vivid. It is still a single experimenter's harness for one parlor game, with rankings that can move as additional models are added.

1. What he built

Blood on the Clocktower is a Werewolf- or Mafia-style game with a good team and an evil team. Evil players know their teammates; good players do not. Each day the table votes to execute someone; each night the demon kills in secret. Evil wins by wiping out the good team; good wins by executing the demon.

Patel says he ranked models by running 368 games with whole teams made of copies of one model versus another. He spent "about $20,000 of API credits" on the harness and matches. He points viewers to clocktowerbench.com for logs, a leaderboard and a way for humans to play the same interface.

2. What he claims the ranking shows

His headline comparison: Claude Opus 4.6 against GPT-6 Astro. Astro is "so far the best model at Blood on the Clocktower," he says, with about a 430 Elo-point gap, meaning Astro wins about 92% of games against Opus 4.6. He also says that between GPT-5.4 and GPT-6 Astra, OpenAI gained about 580 Elo points, enough that Astra wins about 97% of games against 5.4, and that OpenAI's newest models are "way better" than Anthropic's at this game, especially on the good side, where they stay more skeptical of single pieces of evidence.

The video walks through set pieces: an Astro demon bluffing as investigator while a minion volunteers to be sacrificed; an Opus evil "heiress" who replaces a dead demon and steers the village into executing the Saint. Those scenes illustrate coordination and bluffing inside this ruleset. They are selected highlights, not a random sample of all games. The economic angle is simple: if labs compete on "reasoning," a game that forces multi-day consistency under adversarial teammates is a scarce signal — but only for that skill stack, and only inside his harness.

Four cards. Game: Blood on the Clocktower, good vs evil. Method: 368 whole-team matchups. Lead he reports: GPT-6 Astro over Claude Opus 4.6. Limit: one game, one harness, not a general lie detector. Banner: a parlor-game benchmark, not a verdict on AI honesty.
How the experiment is set up, from the video. Labels are ours. Made for Long Talk Short.

He also floated extensions he has not fully run: other Clocktower scripts such as ones with a Lunatic role, whether models improve across many games, mixed teams of two labs, and whether future models will detect teammates by writing style. Mixed-team games on his site, he says, played a bit worse than the average of their members, especially as evil. Those are research notes, not scored leaderboard claims from the 368-game set.

He invites humans to play the AIs on the same site. That is useful for intuition and not a substitute for his automated ratings. Music credit and the Tom Taylor et al. LessWrong post appear in the description; he presents his harness as related work in the same game family.

3. What it is not

This is not a general "AI deception" or safety benchmark. It does not measure lying to users, sandbagging on tests or covert goals. It measures play inside one social game, Trouble Brewing, with Patel's prompts, token limits and team design (every seat on a side is the same model). His site describes a Bradley–Terry / Elo-style fit and notes that neighbours with overlapping uncertainty bands are not reliably ordered. Rankings will change as he adds models. Equal settings matter too: his site says models played at each lab's high reasoning effort with the same token ceiling per turn, which is the right kind of control for a hobbyist harness and still not an independent lab evaluation.

In the video itself he discusses OpenAI and Anthropic models by name. He does not report a Grok or other xAI ranking in this episode. If a later leaderboard update adds one, that would be separate from what this video shows.

Weighing it: what holds up

Six rows. 368 games claimed: from the video. About $20,000 API spend: his figure. Astro best so far: his leaderboard claim. 430 Elo vs Opus 4.6: his calculation. OpenAI better detectives than Anthropic: his reading of results. Proves AIs are generally better liars: not what the design measures.
Claims from the episode versus what the design can support. Made for Long Talk Short.

Checked. Video length, the 368-game method as he states it, named models and percentages as he states them, and the existence of clocktowerbench.com and his GitHub and blog links in the description.

Unverified. We did not re-run the games or audit his Elo fit. Auto captions sometimes say Astro and sometimes Astra for the OpenAI model; we kept his on-screen naming as given. Win rates such as 92% and 97% are his translations of Elo gaps, not independent tallies. His description also says OpenAI's newer models are much better than Anthropic's at this game — a lab-level summary of his own board.

Sources. clocktowerbench.com, GitHub harness, Dwarkesh post, related LessWrong write-up he cites.

Disclosures. Patel built and promotes the Clocktower benchmark and frames the title as "best liar." Patel built and promotes the benchmark. The title's "best liar" framing is punchier than "best at this social-deception game." He also sells the story as entertainment: "If my friends pulled off some of these moves while we were playing, I'd give them a fist bump." OpenAI and Anthropic are the labs he emphasizes on camera; other labs' absence from the on-video ranking is a scope choice, not proof they were tested and lost.

Related reading: Doshi on building ahead of models · AI agents: trust before autonomy

Long Talk Short

  • Patel's experiment is 368 Blood on the Clocktower games with whole teams of one model versus another, not a general deception test.
  • He reports GPT-6 Astro leading, with large Elo gaps over Claude Opus and prior GPT models.
  • OpenAI's edge, in his reading, is mostly better detection on the good team, not only better lying.
  • Highlights of clever bluffs are real plays from his logs; they are not a random sample.
  • No Grok or xAI ranking is reported in this video.

The one thing to remember: Treat a Clocktower leaderboard as a parlor-game benchmark — and do not read a missing lab as a loss it never played.

This summary is for general information only and is not investment advice or a vendor ranking.

Source: Dwarkesh Patel, "Which AI is the best liar?," on YouTube. This is an independent summary and is not affiliated with Dwarkesh Patel.