English
Related papers

Related papers: MementoGUI: Learning Agentic Multimodal Memory Con…

200 papers

Existing memory systems for embodied agents typically inject retrieved memory as static context at episode start, a paradigm we term Ahead-of-time Monolithic Memory Injection (AMMI). However, this static design quickly becomes misaligned…

Robotics · Computer Science 2026-05-15 Xin Ding , Xinrui Wang , Yifan Yang , Hao Wu , Shiqi Jiang , Qianxi Zhang , Liang Mi , Hanxin Zhu , Kun Li , Yunxin Liu , Zhibo Chen , Ting Cao

Large Language Model (LLM) agents increasingly serve as personal assistants and workplace collaborators, where their utility depends on memory systems that extract, retrieve, and apply information across long-running conversations. However,…

Computation and Language · Computer Science 2026-05-19 Jingbo Yang , Kwei-Herng Lai , Xiaowen Wang , Shiyu Chang , Yaar Harari , Evgeniy Gabrilovich

Self-evolving memory systems are unprecedentedly reshaping the evolutionary paradigm of large language model (LLM)-based agents. Prior work has predominantly relied on manually engineered memory architectures to store trajectories, distill…

Computation and Language · Computer Science 2025-12-23 Guibin Zhang , Haotian Ren , Chong Zhan , Zhenhong Zhou , Junhao Wang , He Zhu , Wangchunshu Zhou , Shuicheng Yan

Graphical User Interface (GUI) Agents, powered by large language and vision-language models, hold promise for enabling end-to-end automation in digital environments. However, their progress is fundamentally constrained by the scarcity of…

Machine Learning · Computer Science 2025-09-22 Musen Lin , Minghao Liu , Taoran Lu , Lichen Yuan , Yiwei Liu , Haonan Xu , Yu Miao , Yuhao Chao , Zhaojian Li

Large Language Models (LLMs) have recently been widely adopted in conversational agents. However, the increasingly long interactions between users and agents accumulate extensive dialogue records, making it difficult for LLMs with limited…

Computation and Language · Computer Science 2025-09-30 Derong Xu , Yi Wen , Pengyue Jia , Yingyi Zhang , wenlin zhang , Yichao Wang , Huifeng Guo , Ruiming Tang , Xiangyu Zhao , Enhong Chen , Tong Xu

Large language models (LLMs) excel at many NLP tasks but struggle to sustain long-term interactions due to limited attention over extended dialogue histories. Retrieval-augmented generation (RAG) mitigates this issue but lacks reliable…

Computation and Language · Computer Science 2026-01-23 Chunliang Chen , Ming Guan , Xiao Lin , Jiaxu Li , Luxi Lin , Qiyi Wang , Xiangyu Chen , Jixiang Luo , Changzhi Sun , Dell Zhang , Xuelong Li

AI agents increasingly operate over extended time horizons, yet their ability to retain, organize, and recall multimodal experiences remains a critical bottleneck. Building effective lifelong memory requires navigating a vast design space…

Artificial Intelligence · Computer Science 2026-04-03 Jiaqi Liu , Zipeng Ling , Shi Qiu , Yanqing Liu , Siwei Han , Peng Xia , Haoqin Tu , Zeyu Zheng , Cihang Xie , Charles Fleming , Mingyu Ding , Huaxiu Yao

Graphical User Interface (GUI) Agents have emerged as a transformative paradigm in human-computer interaction, evolving from rule-based automation scripts to sophisticated AI-driven systems capable of understanding and executing complex…

Human-Computer Interaction · Computer Science 2025-06-05 Fei Tang , Haolei Xu , Hang Zhang , Siqi Chen , Xingyu Wu , Yongliang Shen , Wenqi Zhang , Guiyang Hou , Zeqi Tan , Yuchen Yan , Kaitao Song , Jian Shao , Weiming Lu , Jun Xiao , Yueting Zhuang

The advancement of artificial intelligence toward agentic science is currently bottlenecked by the challenge of ultra-long-horizon autonomy, the ability to sustain strategic coherence and iterative correction over experimental cycles…

Artificial Intelligence · Computer Science 2026-03-26 Xinyu Zhu , Yuzhu Cai , Zexi Liu , Bingyang Zheng , Cheng Wang , Rui Ye , Yuzhi Zhang , Linfeng Zhang , Weinan E , Siheng Chen , Yanfeng Wang

Multi-agent systems built on Large Language Models (LLMs) show exceptional promise for complex collaborative problem-solving, yet they face fundamental challenges stemming from context window limitations that impair memory consistency, role…

Artificial Intelligence · Computer Science 2026-01-13 Sizhe Yuen , Francisco Gomez Medina , Ting Su , Yali Du , Adam J. Sobey

Effectively retrieving, reasoning, and understanding multimodal information remains a critical challenge for agentic systems. Traditional Retrieval-augmented Generation (RAG) methods rely on linear interaction histories, which struggle to…

Computer Vision and Pattern Recognition · Computer Science 2026-04-22 Qiuchen Wang , Shihang Wang , Yu Zeng , Qiang Zhang , Fanrui Zhang , Zhuoning Guo , Bosi Zhang , Wenxuan Huang , Lin Chen , Zehui Chen , Pengjun Xie , Ruixue Ding

Human-agent dialogues often exhibit topic continuity-a stable thematic frame that evolves through temporally adjacent exchanges-yet most large language model (LLM) agent memory systems fail to preserve it. Existing designs follow a…

Computation and Language · Computer Science 2026-01-21 Dehao Tao , Guoliang Ma , Yongfeng Huang , Minghu Jiang

Large Language Models (LLMs) are increasingly deployed as long-term interactive agents, yet their limited context windows make it difficult to sustain coherent behavior over extended interactions. Existing memory systems often store…

Artificial Intelligence · Computer Science 2026-01-12 Chuanrui Hu , Xingze Gao , Zuyi Zhou , Dannong Xu , Yi Bai , Xintong Li , Hui Zhang , Tong Li , Chong Zhang , Lidong Bing , Yafeng Deng

With the advancement of vision-language models, web automation has made significant progress. However, deploying autonomous agents in real-world settings remains challenging, primarily due to site heterogeneity, where generalist models lack…

Human-Computer Interaction · Computer Science 2026-04-24 Jihong Wang , Jiamu Zhou , Weiming Zhang , Teng Wang , Weiwen Liu , Zhuosheng Zhang , Xingyu Lou , Weinan Zhang , Huarong Deng , Jun Wang

Large language model (LLM) agents extend generative models with reasoning, tool use, and persistent memory, thereby enabling the automation of complex tasks. In healthcare, such systems could support documentation, care coordination, and…

Artificial Intelligence · Computer Science 2026-03-24 Wenxian Yang , Hanzheng Qiu , Bangqun Zhang , Chengquan Li , Zhiyong Huang , Xiaobin Feng , Rongshan Yu , Jiahong Dong

We explore how reconciling several foundation models (large language models and vision-language models) with a novel unified memory mechanism could tackle the challenging video understanding problem, especially capturing the long-term…

Computer Vision and Pattern Recognition · Computer Science 2024-07-16 Yue Fan , Xiaojian Ma , Rujie Wu , Yuntao Du , Jiaqi Li , Zhi Gao , Qing Li

Graphical user interface (GUI) agents are rapidly progressing toward autonomous interaction and reliable task execution across diverse applications. However, two central challenges remain unresolved: automating the evaluation of agent…

Processing long contexts presents a significant challenge for large language models (LLMs). While recent advancements allow LLMs to handle much longer contexts than before (e.g., 32K or 128K tokens), it is computationally expensive and can…

Computation and Language · Computer Science 2025-04-10 Hongjin Qian , Zheng Liu , Peitian Zhang , Kelong Mao , Defu Lian , Zhicheng Dou , Tiejun Huang

Large reasoning models have demonstrated strong problem-solving abilities, yet real-world tasks often require external tools and long-horizon interactions. Existing agent frameworks typically follow predefined workflows, which limit…

Artificial Intelligence · Computer Science 2026-02-06 Xiaoxi Li , Wenxiang Jiao , Jiarui Jin , Guanting Dong , Jiajie Jin , Yinuo Wang , Hao Wang , Yutao Zhu , Ji-Rong Wen , Yuan Lu , Zhicheng Dou

Utilizing Graphic User Interface (GUI) for human-computer interaction is essential for accessing a wide range of digital tools. Recent advancements in Vision Language Models (VLMs) highlight the compelling potential to develop versatile…

Artificial Intelligence · Computer Science 2025-06-02 Wentong Chen , Junbo Cui , Jinyi Hu , Yujia Qin , Junjie Fang , Yue Zhao , Chongyi Wang , Jun Liu , Guirong Chen , Yupeng Huo , Yuan Yao , Yankai Lin , Zhiyuan Liu , Maosong Sun