English
Related papers

Related papers: UmniBench: Unified Understand and Generation Model…

200 papers

Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration. Existing evaluations either treat the two abilities in isolation or overlook tasks that…

Computer Vision and Pattern Recognition · Computer Science 2026-04-21 Kai Zou , Ziqi Huang , Yuhao Dong , Shulin Tian , Dian Zheng , Hongbo Liu , Jingwen He , Bin Liu , Yu Qiao , Ziwei Liu

Unified multimodal models have recently demonstrated strong generative capabilities, yet whether and when generation improves understanding remains unclear. Existing benchmarks lack a systematic exploration of the specific tasks where…

Computer Vision and Pattern Recognition · Computer Science 2026-03-04 Zimo Wen , Boxiu Li , Wanbo Zhang , Junxiang Lei , Xiaoyu Chen , Yijia Fan , Qi Zhang , Yujiang Wang , Lili Qiu , Bo Li , Ziwei Liu , Caihua Shan , Yifan Yang , Yifei Shen

Unified large multimodal models (LMMs) have achieved remarkable progress in general-purpose multimodal understanding and generation. However, they still operate under a ``one-size-fits-all'' paradigm and struggle to model user-specific…

Computer Vision and Pattern Recognition · Computer Science 2026-01-13 Yu Zhong , Tianwei Lin , Ruike Zhu , Yuqian Yuan , Haoyu Zheng , Liang Liang , Wenqiao Zhang , Feifei Shao , Haoyuan Li , Wanggui He , Hao Jiang , Yueting Zhuang

The advent of Unified Multimodal Models (UMMs) signals a paradigm shift in artificial intelligence, moving from passive perception to active, cross-modal generation. Despite their unprecedented ability to synthesize information, a critical…

Artificial Intelligence · Computer Science 2026-01-15 Jingxuan Wei , Caijun Jia , Xi Bai , Xinglong Xu , Siyuan Li , Linzhuang Sun , Bihui Yu , Conghui He , Lijun Wu , Cheng Tan

Unified multimodal models integrate the reasoning capacity of large language models with both image understanding and generation, showing great promise for advanced multimodal intelligence. However, the community still lacks a rigorous…

Computer Vision and Pattern Recognition · Computer Science 2026-03-24 Hongxiang Li , Yaowei Li , Bin Lin , Yuwei Niu , Yuhang Yang , Xiaoshuang Huang , Jiayin Cai , Xiaolong Jiang , Yao Hu , Long Chen

Recent breakthroughs in large multimodal models (LMMs), such as the impressive GPT-4o-Native, have demonstrated remarkable proficiency in following general-purpose instructions for image generation. However, current benchmarks often lack…

Computer Vision and Pattern Recognition · Computer Science 2025-05-27 Jiayu Wang , Yang Jiao , Yue Yu , Tianwen Qian , Shaoxiang Chen , Jingjing Chen , Yu-Gang Jiang

Existing MLLM benchmarks face significant challenges in evaluating Unified MLLMs (U-MLLMs) due to: 1) lack of standardized benchmarks for traditional tasks, leading to inconsistent comparisons; 2) absence of benchmarks for mixed-modality…

Computer Vision and Pattern Recognition · Computer Science 2025-04-08 Wulin Xie , Yi-Fan Zhang , Chaoyou Fu , Yang Shi , Bingyan Nie , Hongkai Chen , Zhang Zhang , Liang Wang , Tieniu Tan

Multimodal Large Languages models have been progressing from uni-modal understanding toward unifying visual, audio and language modalities, collectively termed omni models. However, the correlation between uni-modal and omni-modal remains…

Computation and Language · Computer Science 2025-10-31 Chen Chen , ZeYang Hu , Fengjiao Chen , Liya Ma , Jiaxing Liu , Xiaoyu Li , Ziwen Wang , Xuezhi Cao , Xunliang Cai

Recent advances in 3D vision have led to specialized models for either 3D understanding (e.g., shape classification, segmentation, reconstruction) or 3D generation (e.g., synthesis, completion, and editing). However, these tasks are often…

Computer Vision and Pattern Recognition · Computer Science 2026-04-21 Peng Huang , Yifeng Chen , Zeyu Zhang , Hao Tang

The emergence of unified multimodal understanding and generation models is rapidly attracting attention because of their ability to enhance instruction-following capabilities while minimizing model redundancy. However, there is a lack of a…

Computer Vision and Pattern Recognition · Computer Science 2025-05-16 Yi Li , Haonan Wang , Qixiang Zhang , Boyu Xiao , Chenchang Hu , Hualiang Wang , Xiaomeng Li

Unified multimodal models (UMMs) aim to integrate multimodal understanding and generation within a unified architecture, yet it remains unclear to what extent their representations are truly aligned across modalities. To investigate this…

Computation and Language · Computer Science 2026-04-08 Cheng Yang , Chufan Shi , Bo Shui , Yaokang Wu , Muzi Tao , Huijuan Wang , Ivan Yee Lee , Yong Liu , Xuezhe Ma , Taylor Berg-Kirkpatrick

In real-world multimodal applications, systems usually need to comprehend arbitrarily combined and interleaved multimodal inputs from users, while also generating outputs in any interleaved multimedia form. This capability defines the goal…

Computer Vision and Pattern Recognition · Computer Science 2026-03-06 Yanlin Li , Minghui Guo , Kaiwen Zhang , Shize Zhang , Yiran Zhao , Haodong Li , Congyue Zhou , Weijie Zheng , Yushen Yan , Shengqiong Wu , Wei Ji , Lei Cui , Furu Wei , Hao Fei , Mong-Li Lee , Wynne Hsu

Unified multimodal models (UMMs) that integrate understanding, reasoning, generation, and editing face inherent trade-offs between maintaining strong semantic comprehension and acquiring powerful generation capabilities. In this report, we…

Unified Multimodal Models (UMMs) integrate both visual understanding and generation within a single framework. Their ultimate aspiration is to create a cycle where understanding and generation mutually reinforce each other. While recent…

Computer Vision and Pattern Recognition · Computer Science 2026-05-26 Zihan Su , Hongyang Wei , Kangrui Cen , Yong Wang , Guanhua Chen , Chun Yuan , Xiangxiang Chu

The integration of visual understanding and generation into unified multimodal models represents a significant stride toward general-purpose AI. However, a fundamental question remains unanswered by existing benchmarks: does this…

Recent advancements in Unified Multimodal Models (UMMs) have enabled remarkable image understanding and generation capabilities. However, while models like Gemini-2.5-Flash-Image show emerging abilities to reason over multiple related…

Computer Vision and Pattern Recognition · Computer Science 2026-02-24 Mingrui Wu , Hang Liu , Jiayi Ji , Xiaoshuai Sun , Rongrong Ji

Recent advances in unified multimodal models (UMM) have demonstrated remarkable progress in both understanding and generation tasks. However, whether these two capabilities are genuinely aligned and integrated within a single model remains…

Computation and Language · Computer Science 2026-02-03 Chenlong Wang , Yuhang Chen , Zhihan Hu , Dongping Chen , Wenhu Chen , Sarah Wiegreffe , Tianyi Zhou

Recent evaluations of Large Multimodal Models (LMMs) have explored their capabilities in various domains, with only few benchmarks specifically focusing on urban environments. Moreover, existing urban benchmarks have been limited to…

Computer Vision and Pattern Recognition · Computer Science 2025-03-11 Baichuan Zhou , Haote Yang , Dairong Chen , Junyan Ye , Tianyi Bai , Jinhua Yu , Songyang Zhang , Dahua Lin , Conghui He , Weijia Li

Although multimodal fusion has made significant progress, its advancement is severely hindered by the lack of adequate evaluation benchmarks. Current fusion methods are typically evaluated on a small selection of public datasets, a limited…

Machine Learning · Computer Science 2026-05-07 Leyan Xue , Changqing Zhang , Kecheng Xue , Xiaohong Liu , Guangyu Wang , Zongbo Han

This paper presents Omni-View, which extends the unified multimodal understanding and generation to 3D scenes based on multiview images, exploring the principle that "generation facilitates understanding". Consisting of understanding model,…

Computer Vision and Pattern Recognition · Computer Science 2026-02-02 JiaKui Hu , Shanshan Zhao , Qing-Guo Chen , Xuerui Qiu , Jialun Liu , Zhao Xu , Weihua Luo , Kaifu Zhang , Yanye Lu
‹ Prev 1 2 3 10 Next ›