SONY RESEARCH INDIA arXiv:2603.16410

ACCEPTED AT LUHME · EMNLP 2026

PlotTwist

Creative plot generation with small language models.

Abhinav Thorat, Ravi Kolla, Jyotin Goel, Madhav Kataria, Niranjan Pedanekar

PAPER ↗ PDF ↗ BIBTEX

STAGE 01 · THE PREMISE

Turning a concise premise into a narrative means satisfying many constraints at once: global coherence, character development, pacing, tone consistency, emotional progression.

Frontier models reach that quality only after preference alignment, and aligning at that scale is prohibitive. PlotTwist externalises narrative structure into explicit training signals instead, so 3B active parameters compete with systems many times larger.

ACTIVE PARAMS
PER TOKEN

NARRATIVE QUALITY
DIMENSIONS

160

PREFERENCE PAIRS
FROM 5,000 PREMISES

8.36

OVERALL JURY MEAN
HIGHEST OF 12 SYSTEMS

0.91

WIN RATE
VS GPT-4.1

CONTRIBUTIONS WHAT IS NEW NEW IN THIS WORKMost plot-generation work scales the writer. PlotTwist keeps the writer small and moves the intelligence outward, into an aspect-level reward model and an independent jury, so a 3B-active model reaches frontier-competitive quality. SELECT A NODE TO PIN

01 · WORKFLOW

Structured workflow using SLMs for plot generation

A modular framework of three specialised components: an aspect rating reward model, a DPO-trained mixture-of-experts plot generator, and an independent agentic evaluation module.

STAGE 02 · THE POSSIBILITIES BRANCH

THE FRAMEWORK WHAT IS NEW NEW IN THIS WORKThe reward model, the generator and the evaluator are three separate models that never share weights or training data. Nothing that scores a plot has any stake in producing it.

The story-generation machine

One premise enters. Candidate plots branch out, a reward model rates every branch across five dimensions, preference optimisation turns the ranking into a twist in the generator's policy, and an independent jury decides what comes out.

SCROLL →

PREMISESetting, genre and thematic constraint, in one line

CANDIDATESBase MoE plus GPT-4.1, Claude Sonnet 4 and Gemini 2.0 Flash, on one premise

REWARD MODELQwen-3-32B, cross-entropy plus Huber loss, scoring five dimensions

DPOThe twist. 160 high-confidence pairs redirect the policy

GENERATORQwen-3-30B-A3B, 3B active per token. The only model that writes

JURYFive jurors, two deliberation rounds, one held-out judge

PLOTJury mean 8.36, the highest of the twelve systems

STAGE 03 · THE TWIST

WHAT IS NEW NEW IN THIS WORKNo reinforcement learning loop, and no reward model at inference. The reward model is used once, offline, to rank candidates into preference pairs. After DPO the generator carries the preference in its weights and runs in a single pass.

What happens when the plot takes a different turn?

ROMANTIC COMEDY · MODERN TECH STARTUP SIX BEATS, ONE BRANCH POINT

SCROLL →

AS WRITTEN AFTER THE TWIST

The twist is not in the story. It is in the policy. Ranking four candidates gives a preference signal, and DPO on 160 pairs redirects the generator, with no reinforcement learning loop and no reward model at inference.

8.03

BASE MoE
BEFORE DPO

→

8.81

AFTER DPO
+0.78 OVERALL

STAGE 04 · THE COMPANY OF SMALL MODELS

THE CAST WHAT IS NEW NEW IN THIS WORKPositive-negative prompting. Every rater scores each aspect twice, once for what works and once for what fails, and the rating is the difference. Asking for one verdict invites the positivity bias LLM judges are known for; asking for both poles cancels it.

Twelve models, one 3B narrator

Large models never write the plot. They rate it before training and judge it after. The model that writes has 3B active parameters per token.

SCROLL →

RATERS · SYNTHETIC ASPECT LABELS, POSITIVE-NEGATIVE PROMPTING

Qwen-2.5-7BQWEN 2.5 · 7B DENSEAlibaba Qwen. One of five raters that score every plot twice per aspect, once for what works and once for what fails.

Llama-3.3-70BLLAMA 3.3 · 70B DENSEMeta. Rater in the ensemble, and separately the largest open-weight baseline in the study at 7.18, and the held-out judge.

Llama-3.1-8BLLAMA 3.1 · 8B DENSEMeta. Rater in the five-model ensemble that builds the synthetic aspect labels.

DeepSeek-14BDEEPSEEK · 14BDeepSeek. Rater in the ensemble. Its R1 distilled sibling appears separately as a 14B baseline scoring 7.60.

Gemma-27BGEMMA · 27BGoogle. Rater, and the model that derives all 5,000 premises from the source plots.

PLOTTWIST FRAMEWORK

TRAINED OFFLINE · SINGLE INFERENCE PASS

Qwen-3-32B

Aspect rating reward model. 4-bit quantised, fine-tuned with cross-entropy plus Huber loss on 5,000 rated plots.

Qwen-3-30B-A3B

The plot generator. Mixture-of-experts, 3B active parameters per token, aligned by DPO. This is the only model that writes.

JURORS · FIVE FAMILIES, ALL DISJOINT FROM THE GENERATOR

GPT-OSS-120BGPT-OSS · 120BOpenAI, open-weight. Juror, served locally at MXFP4 precision, temperature 0.1.

Gemma-4-31BGEMMA 4 · 31BGoogle. Juror at BF16, and the panel’s most conservative scorer: the one juror that favours Claude Sonnet 4 over PlotTwist, at 0.34.

Mistral-Med-3.5MISTRAL MEDIUM 3.5 · 128BMistral. Juror, served locally at Q6_K precision.

Nemotron-3-120BNEMOTRON 3 SUPER · 120B-A12BNVIDIA. Juror, served locally at FP8 precision.

GLM-4.6VGLM 4.6VZhipu. Juror, served locally at FP8 precision.

Llama-3.3-70B · held-out judgeLLAMA 3.3 · 70BMeta. The held-out judge, from a sixth family disjoint from both the panel and the generator. It consolidates contested cells after two rounds of deliberation.

Model	Character development	Tone consistency	Pacing	Narrative coherence	Emotional turning points	Overall
Claude Sonnet 4	8.30 ±.41	8.23 ±.35	8.09 ±.24	8.49 ±.38	8.63 ±.24	8.35 ±.24
Agents' Room	8.15 ±.38	8.15 ±.37	7.97 ±.37	8.40 ±.34	8.57 ±.38	8.25 ±.26
GPT-4.1	7.88 ±.52	8.14 ±.29	7.99 ±.19	8.37 ±.34	8.41 ±.28	8.16 ±.24
Gemini 2.0 Flash	8.01 ±.41	7.98 ±.32	7.91 ±.19	8.27 ±.32	8.40 ±.29	8.11 ±.24
Qwen3-32B	7.66 ±.57	8.10 ±.34	7.62 ±.39	7.83 ±.57	8.28 ±.31	7.90 ±.33
Qwen-2.5-14B	7.41 ±.59	7.53 ±.43	7.52 ±.31	7.69 ±.48	7.95 ±.39	7.62 ±.33
DeepSeek-R1-14B	7.29 ±.66	7.67 ±.42	7.53 ±.31	7.61 ±.48	7.89 ±.41	7.60 ±.35
Mistral Small 24B	7.38 ±.54	7.70 ±.42	7.49 ±.31	7.57 ±.53	7.85 ±.42	7.60 ±.34
Llama-3.3-70B	7.11 ±.69	7.39 ±.55	6.94 ±.56	6.89 ±.75	7.56 ±.53	7.18 ±.48
WizardLM-30B	6.52 ±.80	7.07 ±.55	6.81 ±.55	6.53 ±.70	6.97 ±.62	6.78 ±.51
Phi-4 Mini	6.58 ±.94	6.95 ±1.27	6.27 ±1.55	6.21 ±1.54	6.88 ±1.07	6.58 ±1.18
PlotTwist	8.11 ±.60	8.42 ±.41	8.14 ±.33	8.51 ±.44	8.61 ±.32	8.36 ±.35

Baseline	Mean	PlotTwist win rate	95% CI	p (Holm)
Claude Sonnet 4	8.35	0.563	[0.488, 0.638]	0.13
Agents' Room	8.25	0.734	[0.663, 0.800]	4.4e−9
GPT-4.1	8.16	0.906	[0.863, 0.950]	2.0e−27
Gemini 2.0 Flash	8.11	0.938	[0.900, 0.975]	1.3e−32
Qwen-3-32B	7.90	0.987	[0.969, 1.000]	2.1e−43
Qwen-2.5-14B	7.62	0.950	[0.913, 0.981]	6.4e−35
Mistral Small 24B	7.60	0.994	[0.981, 1.000]	2.4e−45
DeepSeek-R1-14B	7.60	0.994	[0.981, 1.000]	2.4e−45
Llama-3.3-70B	7.18	0.988	[0.969, 1.000]	1.2e−43
WizardLM-30B	6.78	0.994	[0.981, 1.000]	2.4e−45
Phi-4 Mini	6.58	0.994	[0.981, 1.000]	2.4e−45

STAGE 06 · ALTERNATE REALITIES

QUALITATIVE WALKTHROUGH WHAT IS NEW NEW IN THIS WORKOnly 160 of 5,000 premises become training data. A pair survives only when a frontier model wins outright, scores above 8, and beats the runner-up by 0.5. Alignment here is a curation problem, not a data-volume one.

One premise, four futures

Every premise in the curation set branches into candidate plots. The reward model ranks the branches; only a few premises survive the thresholds and become training signal.

PREMISE

Generate Plot of a Romantic Comedy Set in the Modern Tech Startup Era

5,000 premises derived with Gemma-27B
→ 160 high-confidence pairs retained

Rated across all five narrative quality dimensions by the aspect rating reward model; the aspect scores are averaged into one reward per candidate.

Highest reward of the four. It becomes the accepted response of the preference pair only if a frontier model produced it, its score exceeds 8.0, and it beats the next-best candidate by at least 0.5.

Lowest reward of the four, and therefore the rejected response the generator is trained away from.

Second by reward. Ranking across all four candidates gives the ordering B > D > A > C for this premise.

THE UNEXPECTED ENDING

Trained on 160 pairs curated this way, the 3B-active generator wins 90.6% of premises against GPT-4.1 and 73.4% against Agents' Room, from a single model and a single pass.

WHAT IT ACTUALLY WRITES

PREMISE

A dedicated detective investigates a murder that implicates his beloved, forcing him through a web of concealed truths, blackmail, and the lengths one will go to protect a dark secret.

Generated by the PlotTwist MoE
3B active parameters · single pass

Detective Aris Thorne, still scarred by the unsolved murder of his wife Elara, finds his world shattered when Councilman Silas Croft is found in his penthouse with a single, precise stab wound to the heart.

The evidence points damningly toward Lena Voss, his former lover. A rare antique switchblade, identical to one he gifted her years earlier, is found clutched in Croft's hand.

Croft was not merely corrupt. He ran a blackmail network, and the encrypted files reveal he had been threatening Lena with evidence tied to Elara's death. The blackmail was never about Lena. It was about Aris.

Elara had been investigating Croft as a journalist, and was killed in a staged accident when she came too close. At the rain-lashed docks Aris dismantles the network, then chooses love over career and confesses to the murder to protect Lena.

Abridged from Appendix D.

WHAT THIS DOES NOT SHOW

LLM JUDGES, NOT EXPERTS

Plot quality is assessed by language models rather than expert annotators, and LLM judges agree only moderately with expert readers on creative writing.

ONE COMPARISON UNRESOLVED

The comparison with Claude Sonnet 4 remains statistically unresolved at 160 premises, and correlated juror errors reduce the number of effectively independent opinions.

NARROW SCOPE

English, synopsis-style plots, five craft-oriented dimensions. Reward and preference data are built offline with the aid of larger models, and novelty is left to future work.

END OF PATH · THE PAPER

PlotTwist: A Creative Plot Generation Framework with Small Language Models

Abhinav Thorat, Ravi Kolla, Jyotin Goel, Madhav Kataria, Niranjan Pedanekar

Sony Research India · arXiv:2603.16410 [cs.CL]

ACCEPTED AT LUHME · EMNLP 2026

ARXIV ↗ PDF ↗

BIBTEX

@inproceedings{thorat2026plottwist,
  title     = {PlotTwist: A Creative Plot Generation
               Framework with Small Language Models},
  author    = {Thorat, Abhinav and Kolla, Ravi and
               Goel, Jyotin and Kataria, Madhav and
               Pedanekar, Niranjan},
  booktitle = {Proceedings of LUHME at EMNLP 2026},
  year      = {2026}
}

Limitations, full rubrics, all reward-model and juror prompts, and the 113-specification robustness analysis are in the appendices of the paper.

PlotTwist

Structured workflow using SLMs for plot generation

The story-generation machine

What happens when the plot takes a different turn?

Twelve models, one 3B narrator

Scored by the jury, not by us

One premise, four futures

PlotTwist: A Creative Plot Generation Framework with Small Language Models