English
Related papers

Related papers: ScreenSpot-Pro: GUI Grounding for Professional Hig…

200 papers

Enabling intelligent agents to comprehend and interact with 3D environments through natural language is crucial for advancing robotics and human-computer interaction. A fundamental task in this field is ego-centric 3D visual grounding,…

Computer Vision and Pattern Recognition · Computer Science 2025-05-09 Henry Zheng , Hao Shi , Qihang Peng , Yong Xien Chng , Rui Huang , Yepeng Weng , Zhongchao Shi , Gao Huang

Multimodal large language models (MLLMs) demonstrate strong perception and reasoning performance on existing remote sensing (RS) benchmarks. However, most prior benchmarks rely on low-resolution imagery, and some high-resolution benchmarks…

Computer Vision and Pattern Recognition · Computer Science 2025-12-22 Yunkai Dang , Meiyi Zhu , Donghao Wang , Yizhuo Zhang , Jiacheng Yang , Qi Fan , Yuekun Yang , Wenbin Li , Feng Miao , Yang Gao

Multi-modal Large Language Models (MLLMs) have shown remarkable capabilities across a wide range of vision-language tasks. However, due to the restricted input resolutions, MLLMs face significant challenges in precisely understanding and…

Computer Vision and Pattern Recognition · Computer Science 2025-10-27 Lu Zhang , Jiazuo Yu , Haomiao Xiong , Ping Hu , Yunzhi Zhuge , Huchuan Lu , You He

Recent advancements in Multimodal Large Language Models (MLLMs) have generated significant interest in their ability to autonomously interact with and interpret Graphical User Interfaces (GUIs). A major challenge in these systems is…

Computer Vision and Pattern Recognition · Computer Science 2024-12-17 Hai-Ming Xu , Qi Chen , Lei Wang , Lingqiao Liu

In the field of document understanding, significant advances have been made in the fine-tuning of Multimodal Large Language Models (MLLMs) with instruction-following data. Nevertheless, the potential of text-grounding capability within…

Computer Vision and Pattern Recognition · Computer Science 2023-12-18 Yonghui Wang , Wengang Zhou , Hao Feng , Keyi Zhou , Houqiang Li

The rise of Multimodal Large Language Models (MLLMs), renowned for their advanced instruction-following and reasoning capabilities, has significantly propelled the field of visual reasoning. However, due to limitations in their image…

Computer Vision and Pattern Recognition · Computer Science 2024-06-19 Jiaxing Chen , Yuxuan Liu , Dehu Li , Xiang An , Weimo Deng , Ziyong Feng , Yongle Zhao , Yin Xie

The rapid advancement of Multimodal Large Language Models (MLLMs) has enabled browsing agents to acquire and reason over multimodal information in the real world. But existing benchmarks suffer from two limitations: insufficient evaluation…

Computer Vision and Pattern Recognition · Computer Science 2026-03-19 Zhengbo Zhang , Jinbo Su , Zhaowen Zhou , Changtao Miao , Yuhan Hong , Qimeng Wu , Yumeng Liu , Feier Wu , Yihe Tian , Yuhao Liang , Zitong Shan , Wanke Xia , Yi-Fan Zhang , Bo Zhang , Zhe Li , Shiming Xiang , Ying Yan

The manufacturing sector is increasingly adopting Multimodal Large Language Models (MLLMs) to transition from simple perception to autonomous execution, yet current evaluations fail to reflect the rigorous demands of real-world…

Computer Vision and Pattern Recognition · Computer Science 2026-04-14 Xiangru Jian , Hao Xu , Wei Pang , Xinjian Zhao , Chengyu Tao , Qixin Zhang , Xikun Zhang , Chao Zhang , Guanzhi Deng , Alex Xue , Juan Du , Tianshu Yu , Garth Tarr , Linqi Song , Qiuzhuang Sun , Dacheng Tao

Spatial intelligence is essential for multimodal large language models (MLLMs) operating in the complex physical world. Existing benchmarks, however, probe only single-image relations and thus fail to assess the multi-image spatial…

Computer Vision and Pattern Recognition · Computer Science 2026-05-26 Sihan Yang , Runsen Xu , Yiman Xie , Sizhe Yang , Mo Li , Jingli Lin , Chenming Zhu , Xiaochen Chen , Haodong Duan , Xiangyu Yue , Dahua Lin , Tai Wang , Jiangmiao Pang

The reliability of Multimodal Large Language Models (MLLMs) in real-world settings is often undermined by sensitivity to irrelevant or distracting visual context, an aspect not captured by existing evaluation metrics. We introduce the…

Computer Vision and Pattern Recognition · Computer Science 2025-09-30 Hitesh Laxmichand Patel , Amit Agarwal , Srikant Panda , Hansa Meghwani , Karan Dua , Paul Li , Tao Sheng , Sujith Ravi , Dan Roth

Frontier multimodal large language models (MLLMs) have been reported to achieve over 90% accuracy on fine-grained perception benchmarks. However, such scores do not necessarily imply faithful use of visual evidence. Prior studies have…

Computer Vision and Pattern Recognition · Computer Science 2026-05-27 Jingru Chen , Yiming Liu , Mingtao Chen , Sijie Chen , Richeng Xuan , Liang Yang , Zhichao Hu , Fanyang Lu

We introduce VisualQuest, a novel dataset designed to rigorously evaluate multimodal large language models (MLLMs) on abstract visual reasoning tasks that require the integration of symbolic, cultural, and linguistic knowledge. Unlike…

Computer Vision and Pattern Recognition · Computer Science 2026-01-05 Kelaiti Xiao , Liang Yang , Dongyu Zhang , Paerhati Tulajiang , Hongfei Lin

In recent years, multimodal large language models (MLLMs) have made significant strides by training on vast high-quality image-text datasets, enabling them to generally understand images well. However, the inherent difficulty in explicitly…

Computer Vision and Pattern Recognition · Computer Science 2024-07-08 Yuanze Lin , Yunsheng Li , Dongdong Chen , Weijian Xu , Ronald Clark , Philip Torr , Lu Yuan

Recent advances in mobile Graphical User Interface (GUI) agents highlight the growing need for comprehensive evaluation benchmarks. While new online benchmarks offer more realistic testing than offline ones, they tend to focus on the…

Computation and Language · Computer Science 2026-01-30 Qinzhuo Wu , Zhizhuo Yang , Hanhao Li , Pengzhi Gao , Wei Liu , Jian Luan

Existing efforts in building GUI agents heavily rely on the availability of robust commercial Vision-Language Models (VLMs) such as GPT-4o and GeminiProVision. Practitioners are often reluctant to use open-source VLMs due to their…

Computation and Language · Computer Science 2024-10-31 Zhiyong Wu , Zhenyu Wu , Fangzhi Xu , Yian Wang , Qiushi Sun , Chengyou Jia , Kanzhi Cheng , Zichen Ding , Liheng Chen , Paul Pu Liang , Yu Qiao

Beam prediction is an effective approach to reduce training overhead in massive multiple-input multiple-output (MIMO) systems. However, existing beam prediction models still exhibit limited generalization ability in diverse scenarios, which…

Signal Processing · Electrical Eng. & Systems 2025-06-09 Yizhu Zhao , Li Yu , Lianzheng Shi , Jianhua Zhang , Guangyi Liu

AI agents with advanced reasoning and tool use capabilities have demonstrated impressive performance in web browsing for deep search. While existing benchmarks such as BrowseComp evaluate these browsing abilities, they primarily focus on…

Multimodal Large Language Models (MLLMs) have demonstrated impressive progress in single-image grounding and general multi-image understanding. Recently, some methods begin to address multi-image grounding. However, they are constrained by…

Computer Vision and Pattern Recognition · Computer Science 2026-01-09 Shurong Zheng , Yousong Zhu , Hongyin Zhao , Fan Yang , Yufei Zhan , Ming Tang , Jinqiao Wang

While Vision-Language Models (VLMs) have significantly advanced remote sensing interpretation, enabling them to perform complex, step-by-step reasoning remains highly challenging. Recent efforts to introduce Chain-of-Thought (CoT) reasoning…

Computer Vision and Pattern Recognition · Computer Science 2026-05-27 Lang Sun , Ronghao Fu , Zhuoran Duan , Haoran Liu , Xueyan Liu , Bo Yang

GUI grounding is a critical capability for vision-language models (VLMs) that enables automated interaction with graphical user interfaces by locating target elements from natural language instructions. However, grounding on GUI screenshots…

Computer Vision and Pattern Recognition · Computer Science 2026-03-19 Siqi Pei , Liang Tang , Tiaonan Duan , Long Chen , Shuxian Li , Kaer Huang , Yanzhe Jing , Yiqiang Yan , Bo Zhang , Chenghao Jiang , Borui Zhang , Jiwen Lu