English
Related papers

Related papers: Omni-o3: Deep Nested Omnimodal Deduction for Delib…

200 papers

As multimodal content continues to expand at a rapid pace, audio retrieval has emerged as a key enabling technology for media search, content organization, and intelligent assistants. However, most existing benchmarks concentrate on…

Artificial Intelligence · Computer Science 2026-05-07 Honglei Zhang , Yuting Chen , Chenpeng Hu , Siyue Zhang , Yilei Shi

As large-scale models evolve, language instructions are increasingly utilized in multi-modal tasks. Due to human language habits, these instructions often contain ambiguities in real-world scenarios, necessitating the integration of visual…

Computer Vision and Pattern Recognition · Computer Science 2024-10-07 Minheng Ni , Yutao Fan , Lei Zhang , Wangmeng Zuo

The remarkable success of multimodal large language models (MLLMs) has driven advances in multimodal embeddings, yet existing models remain inherently discriminative, limiting their ability to benefit from reasoning-driven generation…

Machine Learning · Computer Science 2026-03-03 Zhibin Lan , Liqiang Niu , Fandong Meng , Jie Zhou , Jinsong Su

Omni-modal language models are intended to jointly understand audio, visual inputs, and language, but benchmark gains can be inflated when visual evidence alone is enough to answer a query. We study whether current omni-modal benchmarks…

Multimedia · Computer Science 2026-05-15 Che Liu , Lichao Ma , Xiangyu Tony Zhang , Yuxin Zhang , Haoyang Zhang , Xuerui Yang , Fei Tian

Multimodal Large Language Models (MLLMs) have shown strong performance in visual and audio understanding when evaluated in isolation. However, their ability to jointly reason over omni-modal (visual, audio, and textual) signals in long and…

Current visual evaluation approaches are typically constrained to a single task. To address this, we propose OmniQuality-R, a unified reward modeling framework that transforms multi-task quality reasoning into continuous and interpretable…

Computer Vision and Pattern Recognition · Computer Science 2025-10-14 Yiting Lu , Fengbin Guan , Yixin Gao , Yan Zhong , Xinge Peng , Jiakang Yuan , Yihao Liu , Bo Zhang , Xin Li , Zhibo Chen , Weisi Lin

Despite the recent progresses, particularly in developing Language Models, there are fundamental challenges and unanswered questions about how such models can continually learn/memorize, self-improve, and find effective solutions. In this…

Machine Learning · Computer Science 2026-01-01 Ali Behrouz , Meisam Razaviyayn , Peilin Zhong , Vahab Mirrokni

Recent advances in large language models (LLMs) have scaled the potential for reasoning and agentic search, wherein models autonomously plan, retrieve, and reason over external knowledge to answer complex queries. However, the iterative…

Information Retrieval · Computer Science 2026-05-13 Sheng Zhang , Junyi Li , Yingyi Zhang , Pengyue Jia , Yichao Wang , Xiaowei Qian , Wenlin Zhang , Maolin Wang , Yong Liu , Xiangyu Zhao

The recent development in multimodal learning has greatly advanced the research in 3D scene understanding in various real-world tasks such as embodied AI. However, most existing studies are facing two common challenges: 1) they are short of…

Computer Vision and Pattern Recognition · Computer Science 2025-08-05 Xueying Jiang , Lewei Lu , Ling Shao , Shijian Lu

In this paper, we address the challenging task of multimodal reasoning by incorporating the notion of ``slow thinking'' into multimodal large language models (MLLMs). Our core idea is that models can learn to adaptively use different levels…

Computer Vision and Pattern Recognition · Computer Science 2026-01-12 Kun Xiang , Zhili Liu , Terry Jingchen Zhang , Yinya Huang , Yunshuang Nie , Kaixin Cai , Yiyang Yin , Runhui Huang , Hanhui Li , Yihan Zeng , Yu-Jie Yuan , Jianhua Han , Lanqing Hong , Hang Xu , Xiaodan Liang

The 1st Cross-Domain EgoCross Challenge at EgoVis, CVPR 2026 evaluates whether multimodal large language models can reason over egocentric videos across surgery, industry, extreme sports, and animal perspective. We achieved second place in…

Computer Vision and Pattern Recognition · Computer Science 2026-05-28 Zixu Li , Zhiwei Chen , Zhiheng Fu , Wenbo Wang , Yupeng Hu , Weili Guan , Liqiang Nie

In this paper, we address the challenging task of multimodal mathematical reasoning by incorporating the ability of "slow thinking" into multimodal large language models (MLLMs). Our core idea is that different levels of reasoning abilities…

Computer Vision and Pattern Recognition · Computer Science 2025-08-04 Kun Xiang , Zhili Liu , Zihao Jiang , Yunshuang Nie , Kaixin Cai , Yiyang Yin , Runhui Huang , Haoxiang Fan , Hanhui Li , Weiran Huang , Yihan Zeng , Yu-Jie Yuan , Jianhua Han , Lanqing Hong , Hang Xu , Xiaodan Liang

While streaming omni-video understanding demands continuous perception and proactive, real-time interaction, this crucial area remains largely under-explored. Current omni-modal methods are inherently designed for offline settings, limiting…

Computer Vision and Pattern Recognition · Computer Science 2026-05-26 Ming Xie , Zizheng Huang , Xudong Tan , Chao Wang , Xiangyu Zeng , Wenxiao Wu , Tao Chen , Limin Wang , Yanwei Fu

Current research in multimodal models faces a key challenge where enhancing generative capabilities often comes at the expense of understanding, and vice versa. We analyzed this trade-off and identify the primary cause might be the…

Computer Vision and Pattern Recognition · Computer Science 2026-04-01 Sen Ye , Mengde Xu , Shuyang Gu , Di He , Liwei Wang , Han Hu

Recent advancements in large language models (LLMs) have exhibited promising performance in solving sequential decision-making problems. By imitating few-shot examples provided in the prompts (i.e., in-context learning), an LLM agent can…

Artificial Intelligence · Computer Science 2024-02-27 Yuchen Xiao , Yanchao Sun , Mengda Xu , Udari Madhushani , Jared Vann , Deepeka Garg , Sumitra Ganesh

In this paper, we introduce OmniEval, a benchmark for evaluating omni-modality models like MiniCPM-O 2.6, which encompasses visual, auditory, and textual inputs. Compared with existing benchmarks, our OmniEval has several distinctive…

Computer Vision and Pattern Recognition · Computer Science 2025-07-01 Yiman Zhang , Ziheng Luo , Qiangyu Yan , Wei He , Borui Jiang , Xinghao Chen , Kai Han

With the rapid progress of artificial intelligence (AI) in multi-modal understanding, there is increasing potential for video comprehension technologies to support professional domains such as medical education. However, existing benchmarks…

Computer Vision and Pattern Recognition · Computer Science 2025-07-08 Shenxi Liu , Kan Li , Mingyang Zhao , Yuhang Tian , Bin Li , Shoujun Zhou , Hongliang Li , Fuxia Yang

Recent advancements in multimodal slow-thinking systems have demonstrated remarkable performance across various visual reasoning tasks. However, their capabilities in text-rich image reasoning tasks remain understudied due to the absence of…

Machine Learning · Computer Science 2026-05-27 Mingxin Huang , Yongxin Shi , Dezhi Peng , Songxuan Lai , Zecheng Xie , Lianwen Jin

Despite their remarkable natural language understanding capabilities, Large Language Models (LLMs) have been underutilized for retrieval tasks. We present Search-R3, a novel framework that addresses this limitation by adapting LLMs to…

Computation and Language · Computer Science 2026-04-10 Yuntao Gui , James Cheng

Autonomous driving world models are expected to work effectively across three core dimensions: state, action, and reward. Existing models, however, are typically restricted to limited state modalities, short video sequences, imprecise…

Computer Vision and Pattern Recognition · Computer Science 2025-11-18 Bohan Li , Zhuang Ma , Dalong Du , Baorui Peng , Zhujin Liang , Zhenqiang Liu , Chao Ma , Yueming Jin , Hao Zhao , Wenjun Zeng , Xin Jin