English
Related papers

Related papers: Seeing, Listening, Remembering, and Reasoning: A M…

200 papers

Complex reasoning in tool-augmented agent frameworks is inherently long-horizon, causing reasoning traces and transient tool artifacts to accumulate and strain the bounded working context of large language models. Without explicit memory…

Artificial Intelligence · Computer Science 2026-01-14 Hongjin Qian , Zhao Cao , Zheng Liu

Agentic AI systems have gained significant attention for their ability to autonomously perform complex tasks. However, their reliance on well-prepared tools limits their applicability in the medical domain, which requires to train…

Computer Vision and Pattern Recognition · Computer Science 2025-02-28 Jinghao Feng , Qiaoyu Zheng , Chaoyi Wu , Ziheng Zhao , Ya Zhang , Yanfeng Wang , Weidi Xie

Long-form multimodal video understanding requires integrating vision, speech, and ambient audio with coherent long-range reasoning. Existing benchmarks emphasize either temporal length or multimodal richness, but rarely both and while some…

Omnimodal large language models have made significant strides in unifying audio and visual modalities; however, they often face challenges in fine-grained cross-modal understanding and have difficulty with multimodal alignment. To address…

Computer Vision and Pattern Recognition · Computer Science 2026-02-06 Keda Tao , Wenjie Du , Bohan Yu , Weiqiang Wang , Jian Liu , Huan Wang

Current Large Language Models (LLMs) are not only limited to some maximum context length, but also are not able to robustly consume long inputs. To address these limitations, we propose ReadAgent, an LLM agent system that increases…

Computation and Language · Computer Science 2024-07-23 Kuang-Huei Lee , Xinyun Chen , Hiroki Furuta , John Canny , Ian Fischer

The reasoning capabilities of LLM (Large Language Model) are widely acknowledged in recent research, inspiring studies on tool learning and autonomous agents. LLM serves as the "brain" of the agent, orchestrating multiple tools for…

Machine Learning · Computer Science 2024-03-26 Xiangyan Liu , Rongxue Li , Wei Ji , Tao Lin

Next-generation visual assistants, such as smart glasses, embodied agents, and always-on life-logging systems, must reason over an entire day or more of continuous visual experience. In ultra-long video settings, relevant information is…

Computer Vision and Pattern Recognition · Computer Science 2026-05-12 Ziyang Wang , Yue Zhang , Shoubin Yu , Ce Zhang , Zengqi Zhao , Jaehong Yoon , Hyunji Lee , Gedas Bertasius , Mohit Bansal

Large reasoning models have demonstrated strong problem-solving abilities, yet real-world tasks often require external tools and long-horizon interactions. Existing agent frameworks typically follow predefined workflows, which limit…

Artificial Intelligence · Computer Science 2026-02-06 Xiaoxi Li , Wenxiang Jiao , Jiarui Jin , Guanting Dong , Jiajie Jin , Yinuo Wang , Hao Wang , Yutao Zhu , Ji-Rong Wen , Yuan Lu , Zhicheng Dou

Intelligent agents need to remember salient information to reason in partially-observed environments. For example, agents with a first-person view should remember the positions of relevant objects even if they go out of view. Similarly, to…

Artificial Intelligence · Computer Science 2022-10-25 Jurgis Pasukonis , Timothy Lillicrap , Danijar Hafner

Large Reasoning Models (LRMs) like o3 and DeepSeek-R1 have achieved remarkable progress in reasoning tasks with long cot. However, they remain computationally inefficient and struggle with accuracy when solving problems requiring complex…

Artificial Intelligence · Computer Science 2026-03-03 Haipeng Luo , Huawen Feng , Qingfeng Sun , Can Xu , Kai Zheng , Yufei Wang , Tao Yang , Han Hu , Yansong Tang

AI agents with advanced reasoning and tool use capabilities have demonstrated impressive performance in web browsing for deep search. While existing benchmarks such as BrowseComp evaluate these browsing abilities, they primarily focus on…

Agent memory shapes how Large Language Model (LLM)-powered agents, akin to the human brain, progressively refine themselves through environment interactions. Existing paradigms remain constrained: parametric memory forcibly adjusts model…

Computation and Language · Computer Science 2025-10-14 Guibin Zhang , Muxin Fu , Shuicheng Yan

Large Language Models (LLMs) are increasingly used as autonomous agents in complex, long-horizon applications, where effective memory is critical for sustained performance. Yet existing memory benchmarks are largely dialogue-centric, while…

Large language model-based agents operating in long-horizon interactions require memory systems that support temporal consistency, multi-hop reasoning, and evidence-grounded reuse across sessions. Existing approaches largely rely on…

Computation and Language · Computer Science 2026-01-27 Juexiang Ye , Xue Li , Xinyu Yang , Chengkai Huang , Lanshun Nie , Lina Yao , Dechen Zhan

Long-term agent memory is increasingly multimodal, yet existing evaluations rarely test whether agents preserve the visual evidence needed for later reasoning. In prior work, many visually grounded questions can be answered using only…

Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents with linear complexity without performance degradation during extrapolation remains the ultimate challenge in long-text…

Computation and Language · Computer Science 2025-07-04 Hongli Yu , Tinghong Chen , Jiangtao Feng , Jiangjie Chen , Weinan Dai , Qiying Yu , Ya-Qin Zhang , Wei-Ying Ma , Jingjing Liu , Mingxuan Wang , Hao Zhou

Existing MLLMs encounter significant challenges in modeling the temporal context within long videos. Currently, mainstream Agent-based methods use external tools to assist a single MLLM in answering long video questions. Despite such…

Computer Vision and Pattern Recognition · Computer Science 2025-12-23 Boyu Chen , Zhengrong Yue , Siran Chen , Zikang Wang , Yang Liu , Peng Li , Yali Wang

Modern language agents must operate over long-horizon, multi-turn interactions, where they retrieve external information, adapt to observations, and answer interdependent queries. Yet, most LLM systems rely on full-context prompting,…

Computation and Language · Computer Science 2025-07-18 Zijian Zhou , Ao Qu , Zhaoxuan Wu , Sunghwan Kim , Alok Prakash , Daniela Rus , Jinhua Zhao , Bryan Kian Hsiang Low , Paul Pu Liang

Recent multimodal LLMs have shown promise in chart-based visual question answering, but their performance declines sharply on unannotated charts-those requiring precise visual interpretation rather than relying on textual shortcuts. To…

Artificial Intelligence · Computer Science 2026-01-08 Rachneet Kaur , Nishan Srishankar , Zhen Zeng , Sumitra Ganesh , Manuela Veloso

To tackle long-context reasoning tasks without the quadratic complexity of standard attention mechanisms, approaches based on agent memory have emerged, which typically maintain a dynamically updated memory when linearly processing document…

Computation and Language · Computer Science 2026-05-12 Baibei Ji , Xiaoyang Weng , Juntao Li , Zecheng Tang , Yihang Lou , Min Zhang