English
Related papers

Related papers: Scaling Spatial Reasoning in MLLMs through Program…

200 papers

Current multimodal large language models (MLLMs) often underperform on mathematical problem-solving tasks that require fine-grained visual understanding. The limitation is largely attributable to inadequate perception of geometric…

Computer Vision and Pattern Recognition · Computer Science 2025-01-14 Shan Zhang , Aotian Chen , Yanpeng Sun , Jindong Gu , Yi-Yu Zheng , Piotr Koniusz , Kai Zou , Anton van den Hengel , Yuan Xue

Scaling LLM-based embodied agents from text-only environments to complex multimodal settings remains a major challenge. Recent work identifies a perception-reasoning-decision gap in standalone Vision-Language Models (VLMs), which often…

Artificial Intelligence · Computer Science 2026-05-08 Mohamed Salim Aissi , Clemence Grislain , Clement Romac , Laure Soulier , Mohamed Chetouani , Olivier Sigaud , Nicolas Thome

Despite rapid progress, embodied agents still struggle with long-horizon manipulation that requires maintaining spatial consistency, causal dependencies, and goal constraints. A key limitation of existing approaches is that task reasoning…

The development of large language models (LLM) has shown progress on reasoning, though studies have largely considered either English or simple reasoning tasks. To address this, we introduce a multilingual structured reasoning and…

Computation and Language · Computer Science 2024-06-13 Bryan Li , Tamer Alkhouli , Daniele Bonadiman , Nikolaos Pappas , Saab Mansour

Large Language Models (LLMs) have advanced recommendation capabilities through enhanced reasoning, but pose significant challenges for real-world deployment due to high inference costs. Conversely, while Small Language Models (SLMs) offer…

Computation and Language · Computer Science 2025-10-13 Prosenjit Biswas , Pervez Shaik , Abhinav Thorat , Ravi Kolla , Niranjan Pedanekar

The recent development of Large Language Models (LLMs) with strong reasoning ability has driven research in various domains such as mathematics, coding, and scientific discovery. Meanwhile, 3D visual grounding, as a fundamental task in 3D…

Computer Vision and Pattern Recognition · Computer Science 2026-01-14 Hsiang-Wei Huang , Kuang-Ming Chen , Wenhao Chai , Cheng-Yen Yang , Jen-Hao Cheng , Jenq-Neng Hwang

Multimodal large language models (MLLMs) are increasingly being applied to spatial cognition tasks, where they are expected to understand and interact with complex environments. Most existing works improve spatial reasoning by introducing…

Computer Vision and Pattern Recognition · Computer Science 2026-05-26 Zhenghao Chen , Huiqun Wang , Di Huang

The lack of reasoning capabilities in Vision-Language Models (VLMs) has remained at the forefront of research discourse. We posit that this behavior stems from a reporting bias in their training data. That is, how people communicate about…

Computation and Language · Computer Science 2026-02-27 Amita Kamath , Jack Hessel , Khyathi Chandu , Jena D. Hwang , Kai-Wei Chang , Ranjay Krishna

When an LLM-based embodied agent fails at a household task, the culprit could be misidentified objects, forgotten sub-goals, or poor action sequencing -- yet existing benchmarks report only a single success rate, making it impossible to…

Robotics · Computer Science 2026-05-13 Yunn Kang Lim , Pengzhan Sun , Ziyi Bai , Xun Xu , Angela Yao , Xulei Yang , Shijie Li

Spatial reasoning focuses on locating target objects based on spatial relations in 3D scenes, which plays a crucial role in developing intelligent embodied agents. Due to the limited availability of 3D scene-language paired data, it is…

Computer Vision and Pattern Recognition · Computer Science 2026-03-27 Shengli Zhou , Minghang Zheng , Feng Zheng , Yang Liu

Vision-Language Models (VLMs) have been applied to autonomous driving to support decision-making in complex real-world scenarios. However, their training on static, web-sourced image-text pairs fundamentally limits the precise…

Computer Vision and Pattern Recognition · Computer Science 2026-01-21 Keishi Ishihara , Kento Sasaki , Tsubasa Takahashi , Daiki Shiono , Yu Yamaguchi

Machine learning (ML) is increasingly being deployed in programmable data planes (switches and SmartNICs) to enable real-time traffic analysis, security monitoring, and in-network decision-making. Decision trees (DTs) are particularly…

Networking and Internet Architecture · Computer Science 2025-09-03 Murayyiam Parvez , Annus Zulfiqar , Roman Beltiukov , Shir Landau Feibish , Walter Willinger , Arpit Gupta , Muhammad Shahbaz

Vision-language models (VLMs) have advanced rapidly, but their ability to capture spatial relationships remains a blindspot. Current VLMs are typically built with contrastive language-image pretraining (CLIP) style image encoders. The…

Computer Vision and Pattern Recognition · Computer Science 2026-01-26 Nahid Alam , Leema Krishna Murali , Siddhant Bharadwaj , Patrick Liu , Timothy Chung , Drishti Sharma , Akshata A , Kranthi Kiran , Wesley Tam , Bala Krishna S Vegesna

Reliable spatial reasoning remains a core bottleneck for vision-language models (VLMs). Existing mainstream training paradigms for spatial reasoning largely rely on outcome alignment or process imitation, lacking explicit constraints on the…

Computer Vision and Pattern Recognition · Computer Science 2026-05-26 Jiangyang Li , Cong Wan , Changjie Wu , Songlin Dong , Lingjun Zhang , Linzhe Shi , Xu Wang , Zhiheng Ma , Hang Zhang , Mu Xu , Yihong Gong

Spatial reasoning, the ability to ground language in 3D understanding, remains a persistent challenge for Vision-Language Models (VLMs). We identify two fundamental bottlenecks: inadequate 3D understanding capabilities stemming from…

Computer Vision and Pattern Recognition · Computer Science 2026-03-06 Yejie Guo , Yunzhong Hou , Wufei Ma , Meng Tang , Ming-Hsuan Yang

Spatial reasoning is a key capability in the field of artificial intelligence, especially crucial in areas such as robotics, computer vision, and natural language understanding. However, evaluating the ability of multimodal large language…

Artificial Intelligence · Computer Science 2025-11-25 Rui Xu , Dakuan Lu , Zicheng Zhao , Xiaoyu Tan , Xintao Wang , Siyu Yuan , Jiangjie Chen , Yinghui Xu

As textual reasoning with large language models (LLMs) has advanced significantly, there has been growing interest in enhancing the multimodal reasoning capabilities of large vision-language models (LVLMs). However, existing methods…

Computer Vision and Pattern Recognition · Computer Science 2025-06-23 Junfei Wu , Jian Guan , Kaituo Feng , Qiang Liu , Shu Wu , Liang Wang , Wei Wu , Tieniu Tan

Spatial Reasoning is an important component of human cognition and is an area in which the latest Vision-language models (VLMs) show signs of difficulty. The current analysis works use image captioning tasks and visual question answering.…

Computation and Language · Computer Science 2025-11-11 Akshar Tumu , Varad Shinde , Parisa Kordjamshidi

Can Vision-Language Models (VLMs) imagine the full scene from just a few views, like humans do? Humans form spatial mental models naturally, internal representations of unseen space, to reason about layout, perspective, and motion. Our…

Vision--language models (VLMs) achieve strong performance on many multimodal benchmarks but remain brittle on spatial reasoning tasks that require aligning abstract overhead representations with egocentric views. We introduce m2sv, a…

Computer Vision and Pattern Recognition · Computer Science 2026-01-28 Yosub Shin , Michael Buriek , Igor Molybog