Existing tabular reasoning benchmarks mostly test models on small, uniform tables, underrepresenting the complexity of real-world data and giving an incomplete view of Large Language Models' (LLMs) reasoning abilities. Real tables are long, heterogeneous, and domain-specific, mixing structured fields with free text and requiring multi-hop reasoning across thousands of tokens. To address this gap, we introduce RUST-BENCH, a benchmark of 7966 questions from 2031 real-world tables spanning two domains: i) RB-Science (NSF grant records) and ii) RB-Sports (NBA statistics). Unlike prior work, RUST-BENCH evaluates LLMs jointly across scale, heterogeneity, domain specificity, and reasoning complexity. Experiments with open-source and proprietary models show that LLMs struggle with heterogeneous schemas and complex multi-hop inference, revealing persistent weaknesses in current architectures and prompting strategies. RUST-BENCH establishes a challenging new testbed for advancing tabular reasoning research.
@article{arxiv.2511.04491,
title = {RUST-BENCH: Benchmarking LLM Reasoning on Unstructured Text within Structured Tables},
author = {Nikhil Abhyankar and Purvi Chaurasia and Sanchit Kabra and Ananya Srivastava and Vivek Gupta and Chandan K. Reddy},
journal= {arXiv preprint arXiv:2511.04491},
year = {2025}
}