English
Related papers

Related papers: EvoCUA: Evolving Computer Use Agents via Learning …

200 papers

Many intelligent user interfaces employ application and user models to determine the user's preferences, goals and likely future actions. Such models require application analysis, adaptation and expansion. Building and maintaining such…

Artificial Intelligence · Computer Science 2013-01-18 Peter J. Gorniak , David L. Poole

As AI agents take on increasingly long-running tasks involving sophisticated planning and execution, there is a corresponding need for novel interaction designs that enable deeper human-agent collaboration. However, most prior works…

Human-Computer Interaction · Computer Science 2026-02-19 K. J. Kevin Feng , Kevin Pu , Matt Latzke , Tal August , Pao Siangliulue , Jonathan Bragg , Daniel S. Weld , Amy X. Zhang , Joseph Chee Chang

Language models are exhibiting increasing capability in knowledge utilization and reasoning. However, when applied as agents in embodied environments, they often suffer from misalignment between their intrinsic knowledge and environmental…

Computation and Language · Computer Science 2024-10-01 Hanlin Wang , Chak Tou Leong , Jian Wang , Wenjie Li

Digital agents capable of automating complex computer tasks have attracted considerable attention due to their immense potential to enhance human-computer interaction. However, existing agent methods exhibit deficiencies in their…

Artificial Intelligence · Computer Science 2024-10-25 Chengyou Jia , Minnan Luo , Zhuohang Dang , Qiushi Sun , Fangzhi Xu , Junlin Hu , Tianbao Xie , Zhiyong Wu

Open-ended image generation is no longer a simple prompt-to-image problem. High-quality generation often requires an agent to combine a model's internal generative ability with external resources. As requests become more diverse and…

Computer Vision and Pattern Recognition · Computer Science 2026-05-25 Sixiang Chen , Zhaohu Xing , Tian Ye , Xinyu Geng , Yunlong Lin , Jianyu Lai , Xuanhua He , Fuxiang Zhai , Jialin Gao , Lei Zhu

Conversational agents (CAs) are increasingly embedded in daily life, yet their ability to navigate user emotions efficiently is still evolving. This study investigates how users with varying traits -- gender, personality, and cultural…

Human-Computer Interaction · Computer Science 2025-11-11 Yuchong Zhang , Yong Ma , Di Fu , Stephanie Zubicueta Portales , Morten Fjeld , Danica Kragic

With the rapid proliferation of large language models and vision-language models, AI agents have evolved from isolated, task-specific systems into autonomous, interactive entities capable of perceiving, reasoning, and acting without human…

Multiagent Systems · Computer Science 2025-10-20 Yuntao Wang , Shaolong Guo , Yanghe Pan , Zhou Su , Fahao Chen , Tom H. Luan , Peng Li , Jiawen Kang , Dusit Niyato

The last few years have witnessed substantial progress in the field of embodied AI where artificial agents, mirroring biological counterparts, are now able to learn from interaction to accomplish complex tasks. Despite this success,…

Computer Vision and Pattern Recognition · Computer Science 2022-01-04 Sarah Pratt , Luca Weihs , Ali Farhadi

We introduce a novel co-design method for autonomous moving agents' shape attributes and locomotion by combining deep reinforcement learning and evolution with user control. Our main inspiration comes from evolution, which has led to wide…

Artificial Intelligence · Computer Science 2022-05-24 Zhiquan Wang , Bedrich Benes , Ahmed H. Qureshi , Christos Mousas

Deep reinforcement learning has proven to be a great success in allowing agents to learn complex tasks. However, its application to actual robots can be prohibitively expensive. Furthermore, the unpredictability of human behavior in…

Robotics · Computer Science 2019-08-16 Mohammad Thabet , Massimiliano Patacchiola , Angelo Cangelosi

Vision-Language-Action models have shown great promise for autonomous driving, yet they suffer from degraded perception after unfreezing the visual encoder and struggle with accumulated instability in long-term planning. To address these…

Computer Vision and Pattern Recognition · Computer Science 2026-05-12 Jiajun Cao , Xiaoan Zhang , Xiaobao Wei , Liyuqiu Huang , Zijian Wang , Hanzhen Zhang , Zhengyu Jia , Wei Mao , Hao Wang , Xianming Liu , Shuchang Zhou , Yang Wang , Shanghang Zhang

Embodied AI agents require a fine-grained understanding of the physical world mediated through visual and language inputs. Such capabilities are difficult to learn solely from task-specific data. This has led to the emergence of pre-trained…

Computer Vision and Pattern Recognition · Computer Science 2024-05-12 Gunshi Gupta , Karmesh Yadav , Yarin Gal , Dhruv Batra , Zsolt Kira , Cong Lu , Tim G. J. Rudner

Most existing spatial reasoning benchmarks focus on static or globally observable environments, failing to capture the challenges of long-horizon reasoning and memory utilization under partial observability and dynamic changes. We introduce…

Computer Vision and Pattern Recognition · Computer Science 2025-11-27 Pukun Zhao , Longxiang Wang , Miaowei Wang , Chen Chen , Fanqing Zhou , Haojian Huang

Embodied AI research is increasingly moving beyond single-task, single-environment policy learning toward multi-task, multi-scene, and multi-model settings. This shift substantially increases the engineering overhead and development time…

Robotics · Computer Science 2026-04-16 Xueyang Zhou , Yihan Sun , Xijie Gong , Guiyao Tie , Pan Zhou , Lichao Sun , Yongchao Chen

In decision support systems, it is essential to get a candidate solution fast, even if it means resorting to an approximation. This constraint introduces a scalability requirement with regard to the kind of heuristics which can be used in…

Multiagent Systems · Computer Science 2014-05-22 D. Krzywicki , Ł. Faber , A. Byrski , M. Kisiel-Dorohinicki

Large language model (LLM) agents accumulate rich episodic trajectories while solving real-world tasks, but it remains unclear whether such experience can be distilled into reusable procedural skills. We introduce SkillEvolBench, a…

Computational agents support humans in many areas of life and are therefore found in heterogeneous contexts. This means they operate in rapidly changing environments and can be confronted with huge state and action spaces. In order to…

Artificial Intelligence · Computer Science 2023-08-31 Nicole Merkle , Ralf Mikut

Despite its promise, imitation learning often fails in long-horizon environments where perfect replication of demonstrations is unrealistic and small errors can accumulate catastrophically. We introduce Cago (Capability-Aware Goal…

Artificial Intelligence · Computer Science 2026-01-14 Yuanlin Duan , Yuning Wang , Wenjie Qiu , He Zhu

Continual Learning (CL) is a powerful tool that enables agents to learn a sequence of tasks, accumulating knowledge learned in the past and using it for problem-solving or future task learning. However, existing CL methods often assume that…

Machine Learning · Computer Science 2025-06-09 Chaofan Pan , Jiafen Liu , Yanhua Li , Linbo Xiong , Fan Min , Wei Wei , Xin Yang

Graphical User Interfaces (GUIs) are central to human-computer interaction, yet automating complex GUI tasks remains a major challenge for autonomous agents, largely due to a lack of scalable, high-quality training data. While recordings of…

Computer Vision and Pattern Recognition · Computer Science 2026-01-13 Yichun Zhang , Xiangwu Guo , Yauhong Goh , Jessica Hu , Zhiheng Chen , Xin Wang , Difei Gao , Mike Zheng Shou