English

BrainBench: Exposing the Commonsense Reasoning Gap in Large Language Models

Artificial Intelligence 2026-03-18 v1 Computation and Language

Abstract

Large language models (LLMs) achieve impressive scores on standard benchmarks yet routinely fail questions that any human would answer correctly in seconds. We introduce BrainBench, a benchmark of 100 brainteaser questions spanning 20 carefully designed categories, each targeting a specific commonsense reasoning failure mode in LLMs. Categories range from implicit physical constraints ("Should I walk or drive my rental car to the return lot?") to semantic scope tricks and default assumption hijacks. We evaluate eight frontier models -- four from the Claude family and four from the GPT family -- using a zero-shot protocol with 10 independent runs per question. The best model, Claude Opus 4.6 with extended thinking, achieves only 80.3% accuracy; the worst, GPT-4o, scores 39.7%. Even top-performing models exhibit a 6-16 percentage-point gap between accuracy and consistency, revealing stochastic reasoning. Cross-lingual evaluation in Chinese shows most models degrade by 2-8 percentage points, confirming that these failures reflect reasoning deficits rather than language-specific artifacts. BrainBench provides a fine-grained diagnostic tool for identifying where and why LLMs substitute surface heuristics for genuine commonsense reasoning.

Keywords

Cite

@article{arxiv.2603.14761,
  title  = {BrainBench: Exposing the Commonsense Reasoning Gap in Large Language Models},
  author = {Yuzhe Tang},
  journal= {arXiv preprint arXiv:2603.14761},
  year   = {2026}
}
R2 v1 2026-07-01T11:21:20.833Z