English
Related papers

Related papers: Benchmarking Spatiotemporal Reasoning in LLMs and …

200 papers

While Large Reasoning Models (LRMs) have achieved remarkable performance by scaling test-time compute, they frequently suffer from Cognitive Inertia, a failure pattern manifesting as either overthinking (inertia of motion) or reasoning…

Machine Learning · Computer Science 2026-02-02 Seojin Lee , ByeongJeong Kim , Hwanhee Lee

Recent advances in large language models (LLMs) have enabled near-human performance on software coding benchmarks, but their effectiveness in RTL code generation remains limited due to the scarcity of high-quality training data. While prior…

Hardware Architecture · Computer Science 2025-07-17 Chenhui Deng , Yun-Da Tsai , Guan-Ting Liu , Zhongzhi Yu , Haoxing Ren

We introduce Cube Bench, a Rubik's-cube benchmark for evaluating spatial and sequential reasoning in multimodal large language models (MLLMs). The benchmark decomposes performance into five skills: (i) reconstructing cube faces from images…

Computation and Language · Computer Science 2025-12-24 Dhruv Anand , Ehsan Shareghi

Multi-modal large language models (MLLMs) have rapidly advanced in visual tasks, yet their spatial understanding remains limited to single images, leaving them ill-suited for physical-world applications that require multi-frame reasoning.…

Computer Vision and Pattern Recognition · Computer Science 2026-05-25 Runsen Xu , Weiyao Wang , Hao Tang , Xingyu Chen , Xiaodong Wang , Fu-Jen Chu , Matt Feiszli , Kevin J. Liang

Large language models (LLMs) have recently shown impressive performance on tasks involving reasoning, leading to a lively debate on whether these models possess reasoning capabilities similar to humans. However, despite these successes, the…

Computation and Language · Computer Science 2024-08-07 Philipp Mondorf , Barbara Plank

Large language models (LLMs) have revolutionized many areas (e.g. natural language processing, software engineering, etc.) by achieving state-of-the-art performance on extensive downstream tasks. Aiming to achieve robust and general…

Artificial Intelligence · Computer Science 2024-01-18 Zhiming Li , Yushi Cao , Xiufeng Xu , Junzhe Jiang , Xu Liu , Yon Shin Teo , Shang-wei Lin , Yang Liu

While large language models (LLMs) excel in mathematical and code reasoning, we observe they struggle with social reasoning tasks, exhibiting cognitive confusion, logical inconsistencies, and conflation between objective world states and…

Computation and Language · Computer Science 2025-10-14 Jialu Du , Guiyang Hou , Yihui Fu , Chen Wu , Wenqi Zhang , Yongliang Shen , Weiming Lu

Structured tables are essential for conveying high-density information in professional domains such as finance, healthcare, and scientific research. Despite the progress in Multimodal Large Language Models (MLLMs), reasoning performance…

Artificial Intelligence · Computer Science 2026-04-07 Xiaoyu Chen , Lu Dai , Hanqing Wang , Zhuoyu Li , Wenbin Dai , Yanzong Zheng , Zhenggang Xia , Junyong Lin , Hui Xiong

Although Large Language Models (LLMs) achieve remarkable performance across various tasks, they often struggle with complex reasoning tasks, such as answering mathematical questions. Recent efforts to address this issue have primarily…

Machine Learning · Computer Science 2024-06-27 Jikun Kang , Xin Zhe Li , Xi Chen , Amirreza Kazemi , Qianyi Sun , Boxing Chen , Dong Li , Xu He , Quan He , Feng Wen , Jianye Hao , Jun Yao

Reasoning ability, a core component of human intelligence, continues to pose a significant challenge for Large Language Models (LLMs) in the pursuit of AGI. Although model performance has improved under the training scaling law, significant…

Machine Learning · Computer Science 2026-01-21 Hengli Li , Chenxi Li , Tong Wu , Xuekai Zhu , Yuxuan Wang , Zhaoxin Yu , Eric Hanchen Jiang , Song-Chun Zhu , Zixia Jia , Ying Nian Wu , Zilong Zheng

Large Language Model (LLM) agents, while proficient in the digital realm, face a significant gap in physical-world deployment due to the challenge of forming and maintaining a robust spatial mental model. We identify three core cognitive…

Artificial Intelligence · Computer Science 2026-01-05 Huan-ang Gao , Zikang Zhang , Tianwei Luo , Kaisen Yang , Xinzhe Juan , Jiahao Qiu , Tianxing Chen , Bingxiang He , Hao Zhao , Hao Zhou , Shilong Liu , Mengdi Wang

Large Reasoning Models (LRMs) represent a breakthrough in AI problem-solving capabilities, but their effectiveness in interactive environments can be limited. This paper introduces and analyzes overthinking in LRMs. A phenomenon where…

Vision-Language Models (VLMs) have recently gained attention due to their competitive performance on multiple downstream tasks, achieved by following user-input instructions. However, VLMs still exhibit several limitations in visual…

Computer Vision and Pattern Recognition · Computer Science 2025-10-23 Simone Alghisi , Gabriel Roccabruna , Massimo Rizzoli , Seyed Mahed Mousavi , Giuseppe Riccardi

Spatial intelligence is crucial for vision--language models (VLMs) in the physical world, yet many benchmarks evaluate largely unconstrained scenes where models can exploit 2D shortcuts. We introduce SSI-Bench, a VQA benchmark for spatial…

Computer Vision and Pattern Recognition · Computer Science 2026-02-10 Chen Yang , Guanxin Lin , Youquan He , Peiyao Chen , Guanghe Liu , Yufan Mo , Zhouyuan Xu , Linhao Wang , Guohui Zhang , Zihang Zhang , Shenxiang Zeng , Chen Wang , Jiansheng Fan

Although the capabilities of large language models have been increasingly tested on complex reasoning tasks, their long-horizon planning abilities have not yet been extensively investigated. In this work, we provide a systematic assessment…

Artificial Intelligence · Computer Science 2026-01-29 Sebastiano Monti , Carlo Nicolini , Gianni Pellegrini , Jacopo Staiano , Bruno Lepri

Large language models (LLMs) and vision language models (VLMs), such as DeepSeek R1,OpenAI o3, and Gemini 2.5 Pro, have demonstrated remarkable reasoning capabilities across logical inference, problem solving, and decision making. However,…

Artificial Intelligence · Computer Science 2025-11-19 Xiaoxing Lian , Aidong Yang , Jun Zhu , Peng Wang , Yue Zhang

Large Language Models (LLMs) still struggle with multi-step logical reasoning. Existing approaches either purely refine the reasoning chain in natural language form or attach a symbolic solver as an external module. In this work, we instead…

Computation and Language · Computer Science 2026-04-22 Feihao Fang , My T. Thai , Yuanyuan Lei

This survey explores recent advancements in reasoning large language models (LLMs) designed to mimic "slow thinking" - a reasoning process inspired by human cognition, as described in Kahneman's Thinking, Fast and Slow. These models, like…

Artificial Intelligence · Computer Science 2025-05-09 Qianjun Pan , Wenkai Ji , Yuyang Ding , Junsong Li , Shilian Chen , Junyi Wang , Jie Zhou , Qin Chen , Min Zhang , Yulan Wu , Liang He

Large language models (LLMs) have achieved remarkable results on tasks framed as reasoning problems, yet their true ability to perform procedural reasoning, executing multi-step, rule-based computations remains unclear. Unlike algorithmic…

Artificial Intelligence · Computer Science 2025-11-20 Mahdi Samiei , Mahdi Mansouri , Mahdieh Soleymani Baghshah

Thinking Large Language Models (LLMs) generate explicit intermediate reasoning traces before final answers, potentially improving transparency, interpretability, and solution accuracy for code generation. However, the quality of these…

Artificial Intelligence · Computer Science 2025-11-11 Haoran Xue , Gias Uddin , Song Wang