English
Related papers

Related papers: GUI-Rise: Structured Reasoning and History Summari…

200 papers

We propose a novel framework for comprehending the reasoning capabilities of large language models (LLMs) through the perspective of meta-learning. By conceptualizing reasoning trajectories as pseudo-gradient descent updates to the LLM's…

Computation and Language · Computer Science 2025-05-27 Junnan Liu , Hongwei Liu , Linchen Xiao , Shudong Liu , Taolin Zhang , Zihan Ma , Songyang Zhang , Kai Chen

We propose a novel framework for summarizing structured enterprise data across multiple dimensions using large language model (LLM)-based agents. Traditional table-to-text models often lack the capacity to reason across hierarchical…

Artificial Intelligence · Computer Science 2025-08-12 Amit Dhanda

Large language models (LLMs) have demonstrated remarkable advances in reasoning capabilities. However, their performance remains constrained by limited access to explicit and structured domain knowledge. Retrieval-Augmented Generation (RAG)…

Computation and Language · Computer Science 2025-10-20 Junlin Wu , Xianrui Zhong , Jiashuo Sun , Bolian Li , Bowen Jin , Jiawei Han , Qingkai Zeng

Recently, Multimodal Large Language Models (MLLMs) have been used as agents to control keyboard and mouse inputs by directly perceiving the Graphical User Interface (GUI) and generating corresponding commands. However, current agents…

Computer Vision and Pattern Recognition · Computer Science 2025-03-25 Dongping Chen , Yue Huang , Siyuan Wu , Jingyu Tang , Liuyi Chen , Yilin Bai , Zhigang He , Chenlong Wang , Huichi Zhou , Yiqiang Li , Tianshuo Zhou , Yue Yu , Chujie Gao , Qihui Zhang , Yi Gui , Zhen Li , Yao Wan , Pan Zhou , Jianfeng Gao , Lichao Sun

Long Chain-of-Thought (LCoT), achieved by Reinforcement Learning with Verifiable Rewards (RLVR), has proven effective in enhancing the reasoning capabilities of Large Language Models (LLMs). However, reasoning in current LLMs is primarily…

GUI task automation streamlines repetitive tasks, but existing LLM or VLM-based planner-executor agents suffer from brittle generalization, high latency, and limited long-horizon coherence. Their reliance on single-shot reasoning or static…

Artificial Intelligence · Computer Science 2025-09-29 Seoyoung Lee , Seonbin Yoon , Seongbeen Lee , Hyesoo Kim , Joo Yong Sim

User simulators serve as the critical interactive environment for agent post-training, and an ideal user simulator generalizes across domains and proactively engages in negotiation by challenging or bargaining. However, current methods…

Computation and Language · Computer Science 2026-01-15 Feng Zhang , Shijia Li , Chunmao Zhang , Zhanyu Ma , Jun Xu , Jiuchong Gao , Jinghua Hao , Renqing He , Jingwen Xu , Han Liu

GUI agents hold significant potential to enhance the experience and efficiency of human-device interaction. However, current methods face challenges in generalizing across applications (apps) and tasks, primarily due to two fundamental…

Computer Vision and Pattern Recognition · Computer Science 2025-03-25 Yuchen Sun , Shanhui Zhao , Tao Yu , Hao Wen , Samith Va , Mengwei Xu , Yuanchun Li , Chongyang Zhang

Strategic reasoning enables agents to cooperate, communicate, and compete with other agents in diverse situations. Existing approaches to solving strategic games rely on extensive training, yielding strategies that do not generalize to new…

Artificial Intelligence · Computer Science 2023-05-31 Kanishk Gandhi , Dorsa Sadigh , Noah D. Goodman

Large language models (LLMs) have demonstrated strong reasoning capabilities, and as existing approaches for enhancing LLM reasoning continue to mature, increasing attention has shifted toward meta-reasoning as a promising direction for…

Artificial Intelligence · Computer Science 2026-04-21 Ziqing Zhuang , Linhai Zhang , Jiasheng Si , Deyu Zhou , Yulan He

Self-consistency methods are the core technique for improving the reasoning reliability of multimodal large language models (MLLMs). By generating multiple reasoning results through repeated sampling and selecting the best answer via…

Computation and Language · Computer Science 2026-02-05 Xinglong Yang , Zhilin Peng , Zhanzhan Liu , Haochen Shi , Sheng-Jun Huang

When reading long-form text, human cognition is complex and structurized. While large language models (LLMs) process input contexts through a causal and sequential perspective, this approach can potentially limit their ability to handle…

Computation and Language · Computer Science 2024-11-01 Kai Liu , Zhihang Fu , Chao Chen , Wei Zhang , Rongxin Jiang , Fan Zhou , Yaowu Chen , Yue Wu , Jieping Ye

A key method for creating Artificial Intelligence (AI) agents is Reinforcement Learning (RL). However, constructing a standalone RL policy that maps perception to action directly encounters severe problems, chief among them being its lack…

GUI prototyping is a fundamental component in the development of modern interactive systems, which are now ubiquitous across diverse application domains. GUI prototypes play a critical role in requirements elicitation by enabling…

Software Engineering · Computer Science 2025-08-06 Kristian Kolthoff , Felix Kretzer , Christian Bartelt , Alexander Maedche , Simone Paolo Ponzetto

Graphical User Interface (GUI) action grounding is a critical step in GUI automation that maps language instructions to actionable elements on GUI screens. Most recent works of GUI action grounding leverage large GUI datasets to fine-tune…

Computation and Language · Computer Science 2025-01-28 Yue Fan , Handong Zhao , Ruiyi Zhang , Yu Shen , Xin Eric Wang , Gang Wu

Large language models (LLMs) have demonstrated strong reasoning and tool-use capabilities, yet they often fail in real-world tool-interactions due to incorrect parameterization, poor tool selection, or misinterpretation of user intent.…

Artificial Intelligence · Computer Science 2025-09-23 Hy Dang , Tianyi Liu , Zhuofeng Wu , Jingfeng Yang , Haoming Jiang , Tao Yang , Pei Chen , Zhengyang Wang , Helen Wang , Huasheng Li , Bing Yin , Meng Jiang

Recent advances in large language models (LLMs) have unlocked powerful reasoning and decision-making capabilities. However, their inherent dependence on static parametric memory fundamentally limits their adaptability, factual accuracy, and…

Information Retrieval · Computer Science 2025-08-07 Xinkui Zhao , Haode Li , Yifan Zhang , Guanjie Cheng , Yueshen Xu

Recent advances in foundation models, particularly Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs), have facilitated the development of intelligent agents capable of performing complex tasks. By leveraging the…

As large language models (LLMs) have demonstrated strong reasoning abilities in structured tasks (e.g., coding and mathematics), we explore whether these abilities extend to strategic multi-agent environments. We investigate strategic…

General Economics · Economics 2025-10-23 Gavin Kader , Dongwoo Lee

Large Language Model (LLM)-based web agents excel at knowledge-intensive tasks but face a fundamental conflict between the need for extensive exploration and the constraints of limited context windows. Current solutions typically rely on…