English
Related papers

Related papers: GAIA: A Data Flywheel System for Training GUI Test…

200 papers

Large-language-model (LLM) agents exhibit complex, context-sensitive behaviour that quickly renders static benchmarks and ad-hoc manual testing obsolete. We present Neo, a configurable, multi-agent framework that automates realistic,…

The predominant approach for training web navigation agents is to gather human demonstrations for a set of popular websites and hand-written tasks, but it is becoming clear that human data is an inefficient resource. We develop a pipeline…

Machine Learning · Computer Science 2025-05-23 Brandon Trabucco , Gunnar Sigurdsson , Robinson Piramuthu , Ruslan Salakhutdinov

Evaluating the quality of multi-turn chatbot interactions remains challenging, as most existing methods assess interactions at the turn level without addressing whether a user's overarching goal was fulfilled. A ``goal'' here refers to an…

Artificial Intelligence · Computer Science 2025-10-07 Deepak Babu Piskala , Sharlene Chen , Udita Patel , Parul Kalra , Rafael Castrillo

Recent developments in large language models (LLMs) have been impressive. However, these models sometimes show inconsistencies and problematic behavior, such as hallucinating facts, generating flawed code, or creating offensive and toxic…

Computation and Language · Computer Science 2024-02-22 Zhibin Gou , Zhihong Shao , Yeyun Gong , Yelong Shen , Yujiu Yang , Nan Duan , Weizhu Chen

In the field of AI-driven human-GUI interaction automation, while rapid advances in multimodal large language models and reinforcement fine-tuning techniques have yielded remarkable progress, a fundamental challenge persists: their…

Computer Vision and Pattern Recognition · Computer Science 2025-10-28 Shaojie Zhang , Ruoceng Zhang , Pei Fu , Shaokang Wang , Jiahui Yang , Xin Du , Shiqi Cui , Bin Qin , Ying Huang , Zhenbo Luo , Jian Luan

Large language models (LLMs) have recently gained much attention in building autonomous agents. However, the performance of current LLM-based web agents in long-horizon tasks is far from optimal, often yielding errors such as repeatedly…

Computation and Language · Computer Science 2025-04-01 Hyungjoo Chae , Namyoung Kim , Kai Tzu-iunn Ong , Minju Gwak , Gwanwoo Song , Jihoon Kim , Sunghwan Kim , Dongha Lee , Jinyoung Yeo

Large Language Models (LLMs) are currently being integrated into industrial software applications to help users perform more complex tasks in less time. However, these LLM-Integrated Applications (LIA) expand the attack surface and…

Cryptography and Security · Computer Science 2025-04-28 Felix Viktor Jedrzejewski , Davide Fucci , Oleksandr Adamov

The paper introduces GUI-Owl-1.5, the latest native GUI agent model that features instruct/thinking variants in multiple sizes (2B/4B/8B/32B/235B) and supports a range of platforms (desktop, mobile, browser, and more) to enable cloud-edge…

Ensuring the long-term reliability of AI models in clinical practice requires continuous performance monitoring and corrective actions when degradation occurs. Addressing this need, this manuscript presents ReclAIm, a multi-agent framework…

Multiagent Systems · Computer Science 2025-10-21 Eleftherios Tzanis , Michail E. Klontzas

Large Language Model (LLM) agents are increasingly improved through interaction, yet most self-evolution methods adapt either the policy or the learning environment in isolation. We identify this structural gap as \emph{Agent-Environment…

Computation and Language · Computer Science 2026-05-26 Yihao Hu , Zhihao Wen , Xiujin Liu , Pan Wang , Xin Zhang , Wei Wu

LLM-based agents represent a paradigm shift in AI, enabling autonomous systems to plan, reason, and use tools while interacting with dynamic environments. This paper provides the first comprehensive survey of evaluation methods for these…

Artificial Intelligence · Computer Science 2026-04-24 Asaf Yehudai , Lilach Eden , Alan Li , Guy Uziel , Yilun Zhao , Roy Bar-Haim , Arman Cohan , Michal Shmueli-Scheuer

The rapid advancement of Large Language Models (LLMs) has created a critical gap in consumer protection due to the lack of standardized certification processes for LLM-powered Artificial Intelligence (AI) systems. This paper argues that…

Computers and Society · Computer Science 2025-02-07 Wesley Pasfield

The growing capabilities of large language models (LLMs) in instruction-following and context-understanding lead to the era of agents with numerous applications. Among these, task planning agents have become especially prominent in…

The ability of large language models (LLMs) to utilize external tools has enabled them to tackle an increasingly diverse range of tasks. However, as the tasks become more complex and long-horizon, the intricate tool utilization process may…

Software Engineering · Computer Science 2025-06-18 Shiting Huang , Zhen Fang , Zehui Chen , Siyu Yuan , Junjie Ye , Yu Zeng , Lin Chen , Qi Mao , Feng Zhao

The rapid advancements in Large Language Models (LLMs) have enabled their deployment as autonomous agents for handling complex tasks in dynamic environments. These LLMs demonstrate strong problem-solving capabilities and adaptability to…

Artificial Intelligence · Computer Science 2025-02-19 Weidi Luo , Shenghong Dai , Xiaogeng Liu , Suman Banerjee , Huan Sun , Muhao Chen , Chaowei Xiao

GUI agents powered by vision-language models (VLMs) show promise in automating complex digital tasks. However, their effectiveness in real-world applications is often limited by scarce training data and the inherent complexity of these…

Computation and Language · Computer Science 2025-09-30 Ran Xu , Kaixin Ma , Wenhao Yu , Hongming Zhang , Joyce C. Ho , Carl Yang , Dong Yu

Tool-augmented Large Language Models (LLMs) have shown impressive capabilities in remote sensing (RS) applications. However, existing benchmarks assume question-answering input templates over predefined image-text data pairs. These…

Computation and Language · Computer Science 2024-05-03 Simranjit Singh , Michael Fore , Dimitrios Stamoulis

The rapid advancement of large Vision-Language Models (VLMs) has propelled the development of pure-vision-based GUI Agents, capable of perceiving and operating Graphical User Interfaces (GUI) to autonomously fulfill user instructions.…

Artificial Intelligence · Computer Science 2025-05-30 Chenyu Yang , Shiqian Su , Shi Liu , Xuan Dong , Yue Yu , Weijie Su , Xuehui Wang , Zhaoyang Liu , Jinguo Zhu , Hao Li , Wenhai Wang , Yu Qiao , Xizhou Zhu , Jifeng Dai

Large language models (LLMs) have demonstrated remarkable performance across a wide range of tasks and domains, with data playing a central role in enabling these advances. Despite this success, the preparation and effective utilization of…

Computation and Language · Computer Science 2026-03-17 Hao Liang , Zhengyang Zhao , Zhaoyang Han , Meiyi Qiang , Xiaochen Ma , Bohan Zeng , Qifeng Cai , Zhiyu Li , Linpeng Tang , Weinan E , Wentao Zhang

Agent benchmarks remain largely English-centric, while their multilingual versions are often built with machine translation (MT) and limited post-editing. We argue that, for agentic tasks, this minimal workflow can easily break benchmark…

Computation and Language · Computer Science 2026-04-29 Yunsu Kim , Kaden Uhlig , Joern Wuebker