English
Related papers

Related papers: Agentic Learner with Grow-and-Refine Multimodal Se…

200 papers

Effectively retrieving, reasoning, and understanding multimodal information remains a critical challenge for agentic systems. Traditional Retrieval-augmented Generation (RAG) methods rely on linear interaction histories, which struggle to…

Computer Vision and Pattern Recognition · Computer Science 2026-04-22 Qiuchen Wang , Shihang Wang , Yu Zeng , Qiang Zhang , Fanrui Zhang , Zhuoning Guo , Bosi Zhang , Wenxuan Huang , Lin Chen , Zehui Chen , Pengjun Xie , Ruixue Ding

Recently, Multimodal Large Language Models (MLLMs) have demonstrated significant potential in complex visual tasks through the integration of Chain-of-Thought (CoT) reasoning. However, in Video Question Answering, extended thinking…

Computer Vision and Pattern Recognition · Computer Science 2026-03-18 Xiaokun Sun , Yubo Wang , Haoyu Cao , Linli Xu

Existing memory-augmented LLM agents often treat memory as a static repository with pre-defined representations and fixed retrieval pipelines, which is brittle in dynamic agentic environments where feedback, task variation, and…

Memory is essential for large vision-language models (LVLMs) to handle long, multimodal interactions, with two method directions providing this capability: long-context LVLMs and memory-augmented agents. However, no existing benchmark…

Computer Vision and Pattern Recognition · Computer Science 2026-05-15 Xiyu Ren , Zhaowei Wang , Yiming Du , Zhongwei Xie , Chi Liu , Xinlin Yang , Haoyue Feng , Wenjun Pan , Tianshi Zheng , Baixuan Xu , Zhengnan Li , Yangqiu Song , Ginny Wong , Simon See

Multimodal Large Language Models (MLLMs) have recently been applied to universal multimodal retrieval, where Chain-of-Thought (CoT) reasoning improves candidate reranking. However, existing approaches remain largely language-driven, relying…

Computer Vision and Pattern Recognition · Computer Science 2026-02-26 Dongyang Chen , Chaoyang Wang , Dezhao Su , Xi Xiao , Zeyu Zhang , Jing Xiong , Qing Li , Yuzhang Shang , Shichao Kan

Recent benchmarks for Large Language Model (LLM) agents mainly evaluate reasoning, planning, and execution. However, memory is also essential for agents, as it enables them to store, update, and retrieve information over time. This ability…

Computation and Language · Computer Science 2026-05-19 Yuyao Wang , Zhongjian Zhang , Mo Chi , Kaichi Yu , Yuhan Li , Miao Peng , Bing Tong , Chen Zhang , Yan Zhou , Jia Li

Effective memory management is essential for large language model (LLM) agents handling long-term interactions. Current memory frameworks typically treat agents as passive "recorders" and retrieve information without understanding its…

Computation and Language · Computer Science 2026-03-03 Xiaohui Zhang , Zequn Sun , Chengyuan Yang , Yaqin Jin , Yazhong Zhang , Wei Hu

Large Language Models (LLMs) based agents excel at diverse tasks, yet they suffer from brittle procedural memory that is manually engineered or entangled in static parameters. In this work, we investigate strategies to endow agents with a…

Computation and Language · Computer Science 2026-04-16 Runnan Fang , Yuan Liang , Xiaobin Wang , Jialong Wu , Shuofei Qiao , Pengjun Xie , Fei Huang , Huajun Chen , Ningyu Zhang

Procedural memory enables large language model (LLM) agents to internalize "how-to" knowledge, theoretically reducing redundant trial-and-error. However, existing frameworks predominantly suffer from a "passive accumulation" paradigm,…

Artificial Intelligence · Computer Science 2026-04-16 Zouying Cao , Jiaji Deng , Li Yu , Weikang Zhou , Zhaoyang Liu , Bolin Ding , Hai Zhao

When faced with complex problems, we tend to engage in slower, more deliberate thinking. In contrast, for simple questions we give quick, intuitive responses. This dual-system thinking approach allows us to allocate cognitive resources…

Computer Vision and Pattern Recognition · Computer Science 2026-03-10 Chenyu Lin , Cheng Chi , Jinlin Wu , Sharon Li , Kaiyang Zhou

Multimodal large language models (MLLMs) have shown strong performance on offline video understanding, but most are limited to offline inference or have weak online reasoning, making multi-turn interaction over continuously arriving video…

Computer Vision and Pattern Recognition · Computer Science 2026-03-13 Lu Wang , Zhuoran Jin , Yupu Hao , Yubo Chen , Kang Liu , Yulong Ao , Jun Zhao

Multimodal large language models are increasingly deployed as long-horizon agents, where memory must do more than recall: it must track an evolving world, revise what has gone stale, and surface the right evidence at decision time. Existing…

Large language model (LLM)-powered multi-agent systems (MAS) demonstrate remarkable collective intelligence, wherein multi-agent memory serves as a pivotal mechanism for continual adaptation. However, existing multi-agent memory designs…

Computation and Language · Computer Science 2026-03-10 Muxin Fu , Xiangyuan Xue , Yafu Li , Zefeng He , Siyuan Huang , Xiaoye Qu , Yu Cheng , Yang Yang

Long video understanding requires more than large context windows. It also needs a memory mechanism that decides what visual evidence to retain, keeps it searchable over long horizons, and grounds later reasoning in recoverable observations…

Computer Vision and Pattern Recognition · Computer Science 2026-05-19 Aiden Yiliu Li , Nels Numan , Anthony Steed

Recent advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced cross-modal understanding and reasoning by incorporating Chain-of-Thought (CoT) reasoning in the semantic space. Building upon this, recent studies…

Computer Vision and Pattern Recognition · Computer Science 2026-04-10 Chengzhi Liu , Yuzhe Yang , Yue Fan , Qingyue Wei , Sheng Liu , Xin Eric Wang

Agents utilizing tools powered by large language models (LLMs) or vision-language models (VLMs) have demonstrated remarkable progress in diverse tasks across text and visual modalities. Unlike traditional tools such as calculators, which…

Computation and Language · Computer Science 2025-10-09 Yunzhong Xiao , Yangmin Li , Hewei Wang , Yunlong Tang , Zora Zhiruo Wang

To support long-term interaction in complex environments, LLM agents require memory systems that manage historical experiences. Existing approaches either retain full interaction histories via passive context extension, leading to…

Artificial Intelligence · Computer Science 2026-01-30 Jiaqi Liu , Yaofeng Su , Peng Xia , Siwei Han , Zeyu Zheng , Cihang Xie , Mingyu Ding , Huaxiu Yao

LLM agents that operate over long context depend on external memory to accumulate knowledge over time. However, existing methods typically store each observation as a single deterministic conclusion (e.g., inferring "API~X failed" from…

Artificial Intelligence · Computer Science 2026-05-11 Junfeng Liao , Qizhou Wang , Jianing Zhu , Bo Du , Rui Yan , Xiuying Chen

Achieving self-evolution in intelligent agents requires the continual accumulation of new knowledge across changing task sequences without forgetting previously acquired abilities. Existing approaches either internalize knowledge by…

Machine Learning · Computer Science 2026-05-22 Dianzhi Yu , Vireo Zhang , Hongru Wang , Yanyu Chen , Minda Hu , Wanghan Xu , Siki Chen , Philip Torr , Zhenfei Yin , Irwin King

Vision-language navigation (VLN) is a critical domain within embedded intelligence, requiring agents to navigate 3D environments based on natural language instructions. Traditional VLN research has focused on improving environmental…

Artificial Intelligence · Computer Science 2024-09-24 Zhiyuan Li , Yanfeng Lv , Ziqin Tu , Di Shang , Hong Qiao