How Clued up are LLMs? Evaluating Multi-Step Deductive Reasoning in a Text-Based Game Environment
Artificial Intelligence
2026-03-19 v1 Computation and Language
Abstract
Deducing whodunit proves challenging for LLM agents. In this paper, we implement a text-based multi-agent version of the classic board game Clue as a rule-based testbed for evaluating multi-step deductive reasoning, with six agents drawn from GPT-4o-mini and Gemini-2.5-Flash. We further investigate whether fine-tuning on structured logic puzzles transfers to improved in-game reasoning and gameplay. Across 18 simulated games, agents achieve only four correct wins, indicating difficulty in maintaining consistent deductive reasoning over the course of a full game. Additionally, we find that fine-tuning does not reliably improve performance and, in some cases, appears to increase reasoning volume without improving reasoning precision.
Keywords
Cite
@article{arxiv.2603.17169,
title = {How Clued up are LLMs? Evaluating Multi-Step Deductive Reasoning in a Text-Based Game Environment},
author = {Rebecca Ansell and Autumn Toney-Wails},
journal= {arXiv preprint arXiv:2603.17169},
year = {2026}
}