English
Related papers

Related papers: Optimizing Multimodal LLMs for Egocentric Video Un…

200 papers

While video large language models (Video-LLMs) excel in understanding slow-paced, real-world egocentric videos, their capabilities in high-velocity, information-dense virtual environments remain under-explored. Existing benchmarks focus on…

Computer Vision and Pattern Recognition · Computer Science 2026-04-21 Jianzhe Ma , Zhonghao Cao , Shangkui Chen , Yichen Xu , Wenxuan Wang , Qin Jin

Accurate and efficient Video Quality Assessment (VQA) has long been a key research challenge. Current mainstream VQA methods typically improve performance by pretraining on large-scale classification datasets (e.g., ImageNet, Kinetics-400),…

Computer Vision and Pattern Recognition · Computer Science 2025-10-10 Yachun Mi , Yu Li , Yanting Li , Chen Hui , Tong Zhang , Zhixuan Li , Chenyue Song , Wei Yang Bryan Lim , Shaohui Liu

There has been growing sentiment recently that modern large multimodal models (LMMs) have addressed most of the key challenges related to short video comprehension. As a result, both academia and industry are gradually shifting their…

Computer Vision and Pattern Recognition · Computer Science 2024-10-04 Jianrui Zhang , Mu Cai , Yong Jae Lee

Large language models (LLMs) deliver impressive results for a variety of tasks, but state-of-the-art systems require fast GPUs with large amounts of memory. To reduce both the memory and latency of these systems, practitioners quantize…

Computer Vision and Pattern Recognition · Computer Science 2026-01-22 Gautom Das , Vincent La , Ethan Lau , Abhinav Shrivastava , Matthew Gwilliam

Multimodal large language models (MLLMs) are flourishing, but mainly focus on images with less attention than videos, especially in sub-fields such as prompt engineering, video chain-of-thought (CoT), and instruction tuning on videos.…

Computer Vision and Pattern Recognition · Computer Science 2024-07-09 Yan Wang , Yawen Zeng , Jingsheng Zheng , Xiaofen Xing , Jin Xu , Xiangmin Xu

Recently, with the emergence of recent Multimodal Large Language Model (MLLM) technology, it has become possible to exploit its video understanding capability on different classification tasks. In practice, we face the difficulty of huge…

Computer Vision and Pattern Recognition · Computer Science 2025-07-04 Xin Dong , Sen Jia , Ming Rui Wang , Yan Li , Zhenheng Yang , Bingfeng Deng , Hongyu Xiong

This technical report presents our solution, EgoAdapt (Egocentric Adaptation via Category, Calibration, and Consistency), to the CVPR 2026 HD-EPIC VQA challenge. HD-EPIC evaluates whether a vision-language model can reason over realistic…

Computer Vision and Pattern Recognition · Computer Science 2026-05-26 Zhiwei Chen , Yupeng Hu , Zixu Li , Zhiheng Fu , Guozhi Qiu , Weili Guan , Liqiang Nie

As large vision language models (VLMs) advance, their capabilities in multilingual visual question answering (mVQA) have significantly improved. Chain-of-thought (CoT) reasoning has been proven to enhance interpretability and complex…

Computer Vision and Pattern Recognition · Computer Science 2026-04-15 Jing Huang , Zhiya Tan , Shutao Gong , Fanwei Zeng , Joey Tianyi Zhou , Changtao Miao , Huazhe Tan , Weibin Yao , Jianshu Li

Open-source multimodal large language models (MLLMs) have shown significant potential in a broad range of multimodal tasks. However, their reasoning capabilities remain constrained by existing instruction-tuning datasets, which were…

Computation and Language · Computer Science 2025-06-05 Jarvis Guo , Tuney Zheng , Yuelin Bai , Bo Li , Yubo Wang , King Zhu , Yizhi Li , Graham Neubig , Wenhu Chen , Xiang Yue

Multimodal large language models have recently achieved remarkable progress in video question answering (VideoQA) by jointly processing visual, textual, and audio information. However, it remains unclear which video representations are most…

Information Retrieval · Computer Science 2025-10-15 Zhi Li , Yanan Wang , Hao Niu , Julio Vizcarra , Masato Taya

Despite significant advances in Multimodal Large Language Models (MLLMs), understanding complex temporal dynamics in videos remains a major challenge. Our experiments show that current Video Large Language Model (Video-LLM) architectures…

Computer Vision and Pattern Recognition · Computer Science 2025-10-31 Ali Rasekh , Erfan Bagheri Soula , Omid Daliran , Simon Gottschalk , Mohsen Fayyaz

Visual Query Localization on long-form egocentric videos requires spatio-temporal search and localization of visually specified objects and is vital to build episodic memory systems. Prior work develops complex multi-stage pipelines that…

Computer Vision and Pattern Recognition · Computer Science 2023-06-16 Hanwen Jiang , Santhosh Kumar Ramakrishnan , Kristen Grauman

Emerging multimodal large language models (MLLMs) exhibit great potential for chart question answering (CQA). Recent efforts primarily focus on scaling up training datasets (i.e., charts, data tables, and question-answer (QA) pairs) through…

Computer Vision and Pattern Recognition · Computer Science 2024-08-13 Xingchen Zeng , Haichuan Lin , Yilin Ye , Wei Zeng

Multimodal Large Language Models (MLLMs) have demonstrated impressive performance in short video understanding. However, understanding long-form videos still remains challenging for MLLMs. This paper proposes TimeSuite, a collection of new…

Computer Vision and Pattern Recognition · Computer Science 2025-02-13 Xiangyu Zeng , Kunchang Li , Chenting Wang , Xinhao Li , Tianxiang Jiang , Ziang Yan , Songze Li , Yansong Shi , Zhengrong Yue , Yi Wang , Yali Wang , Yu Qiao , Limin Wang

Reinforcement learning based post-training paradigms for Video Large Language Models (VideoLLMs) have achieved significant success by optimizing for visual-semantic tasks such as captioning or VideoQA. However, while these approaches…

Computer Vision and Pattern Recognition · Computer Science 2026-01-08 Xiaokun Sun , Zezhong Wu , Zewen Ding , Linli Xu

Large Multimodal Models (LMMs) have demonstrated impressive performance in short video understanding tasks but face great challenges when applied to long video understanding. In contrast, Large Language Models (LLMs) exhibit outstanding…

Computer Vision and Pattern Recognition · Computer Science 2024-10-03 Hongchen Wei , Zhenzhong Chen

The advent of large vision-language models (LVLMs) has spurred research into their applications in multi-modal contexts, particularly in video understanding. Traditional VideoQA benchmarks, despite providing quantitative metrics, often fail…

Computer Vision and Pattern Recognition · Computer Science 2024-10-31 Xinyu Fang , Kangrui Mao , Haodong Duan , Xiangyu Zhao , Yining Li , Dahua Lin , Kai Chen

Current Multimodal Large Language Models (MLLMs) may struggle with understanding long or complex videos due to computational demands at test time, lack of robustness, and limited accuracy, primarily stemming from their feed-forward…

Computer Vision and Pattern Recognition · Computer Science 2025-06-10 Jiahao Meng , Shuyang Sun , Yue Tan , Lu Qi , Yunhai Tong , Xiangtai Li , Longyin Wen

Egocentric video reasoning centers on an unobservable agent behind the camera who dynamically shapes the environment, requiring inference of hidden intentions and recognition of fine-grained interactions. This core challenge limits current…

Computer Vision and Pattern Recognition · Computer Science 2025-10-28 Baoqi Pei , Yifei Huang , Jilan Xu , Yuping He , Guo Chen , Fei Wu , Yu Qiao , Jiangmiao Pang

Multimodal large language models have become a popular topic in deep visual understanding due to many promising real-world applications. However, hour-long video understanding, spanning over one hour and containing tens of thousands of…

Computer Vision and Pattern Recognition · Computer Science 2025-05-14 Heqing Zou , Tianze Luo , Guiyang Xie , Victor Xiao Jie Zhang , Fengmao Lv , Guangcong Wang , Junyang Chen , Zhuochen Wang , Hansheng Zhang , Huaijian Zhang