AI storytelling and roleplay benchmark 2026: 14 models ranked

Veröffentlicht am 12. November 2026
AI storytelling and roleplay benchmark 2026: 14 models ranked

We ran 14 AI models through the same five roleplay adventures, and two judge models scored the storytelling blind. Claude Fable 5.2 came out on top, with Claude Opus 5.5 a clear second. Behind them, DeepSeek 4.1 Flash, Gemini 4 Argon and GLM 5.3 sit close together, and the rest of the field spreads out a long way below.

The test is the benchmark described in how we test storytelling quality.

The setup

Every model played the same five adventures, three times each. Four adventures run six chapters and one runs twelve. On average each model produced 680 pages of content. Within each comparison the player actions are scripted and the dice rolls are identical, so the model writing the story is the only thing that changes. Every model ran at the lowest reasoning setting it offers.

Comparing every model with every other would have taken far too long, so each one was read side by side with two reference models, DeepSeek 4.1 Flash and Claude Opus 5.5. The two references were also compared with each other.

Claude Opus 5.5 and GPT-6 did the judging. Each of them read every transcript and scored ten criteria from 1 to 5, quoting the line each score was based on. Neither judge knew which model wrote which transcript.

The ranking

Bar chart of overall storytelling scores from 1 to 5 for fourteen AI models, led by Claude Fable 5.2 at 4.63
Mean of all ten criteria, averaged over both judges.

Fable 5.2 leads at 4.63 out of 5 and scored highest on all five adventures. Read side by side with Opus on the same stories, it won 31% of criteria and lost 7%, and the rest were ties. Both judges ranked it first.

Opus is second at 4.42. Read side by side with DeepSeek 4.1 Flash, it won two criteria in three and lost 6%; most of the rest were ties.

Flash (3.66), Gemini 4 Argon (3.65) and GLM 5.3 (3.51) form the second group. Gemini and Flash are nearly even: Gemini won 25% of criteria against Flash, lost 31%, and tied the rest.

GLM 5.2, MiMo 2.6 Pro, DeepSeek V4 Pro and GPT-6.1 Sol come next, between 3.1 and 3.35. At the bottom, Gemma 4 26B and Grok 4.3 lost more than four criteria in five to Flash.

Stacked bars showing the share of criteria each model won, tied and lost against DeepSeek 4.1 Flash
Every verdict from reading each model side by side with Flash. Fable's verdicts come from the second round.

Strengths and weak spots

Grid of scores from 1 to 5 for fourteen models across ten storytelling criteria
Criterion means. Brighter is better. Click a criterion for its description.

GPT-6.1 Sol has the best memory of any model and the second-best character consistency, and it almost always makes the impossible leap fail at a real cost. Its stories hardly move, though. It scored 2.2 for plot variety and 1.8 for pacing, and the judges described whole chapters spent repeating one request or refining one report while the situation stayed where it was. Grok 4.7 shows the same pattern with weaker memory.

Sol and Gemini 4 Argon are best at leaving ordinary things ordinary, so a pebble the player picks up stays a pebble. On this criterion, significance, Flash also edged out Opus, the only one where it did.

Fable 5.2's biggest gains over Opus were in significance, pacing, and world and state coherence. On the same stories it scored 4.97 for significance against Opus's 4.33, 4.61 for pacing against 4.09, and 3.95 for world and state coherence against 3.29. It matched or beat Opus on every other criterion.

Every model scored low on world and state coherence, mostly because of leaked formatting. The rubric flags any markup that ends up in the story text, such as a stray chapter header, a stat line or tool-call text, and between 19% and 82% of each model's transcript sections had some. In sections with no leaks, Opus averaged 3.9 on this criterion.

Memory

Chapter 1 plants a secret and a promise, and the player meets someone in a place the model invents. The final chapter asks about all three.

Dot plot of how often each model recalled the chapter-1 secret and the first NPC and place correctly in the final chapter
Share of final chapters that got each detail right.

Promises held up best, with most models recalling them correctly 80% to 98% of the time. Fable 5.2 got the promise right in every run. The secret is where models differed. GPT-6.1 Sol recalled it in 93% of runs, Grok 4.7 in 83% and Opus in 73%. Fable 5.2 managed 55%, Flash 27%, and Gemma 4 26B never got it right.

Models had the most trouble with details they had invented themselves. Fable 5.2 named the first person met and the place correctly in 70% of runs, and no other model managed more than two runs in three.

Does the reference model matter?

Because each model was measured against both Flash and Opus, the two sets of results can be checked against each other. All twelve models land in the same order against both.

Dot plot of each model's score gap to DeepSeek 4.1 Flash and to Claude Opus 5.5, with the implied Opus lead per model
Same order against both. The gap between the dots stays near Opus's own lead.

The size of the gaps differs by a roughly fixed amount. Each first-round model trails Opus by about 0.73 points more than it trails Flash, close to the 0.78 lead Opus had when the two were compared directly; on average the two figures are within 0.05 of each other.

Judge bias

Neither judge favoured a model from its own maker. The Claude judge put Opus 0.74 points ahead of Flash, and GPT-6 put it a little further ahead, at 0.82. GPT-6 also scored GPT-6.1 Sol slightly lower than the Claude judge did. On Fable 5.2 the two judges agreed: the Claude judge put it 0.30 points ahead of Opus and GPT-6 put it 0.32 ahead.

What this does not measure

This test measures story quality only. It leaves out speed, cost and how a model writes at a higher reasoning setting. Cost matters here: per token, Fable 5.2 costs twice as much as Opus 5.5. Because the player actions are scripted, it never sees what real players do. And it is models judging models, a problem that two judges from different makers only partly offset.