通过模型、数据和测试时扩展提升开源多模态模型性能边界
计算机视觉与模式识别
2025-09-29 v5
摘要
我们介绍InternVL 2.5，一个高级多模态大型语言模型（MLLM）系列，基于 InternVL 2.0 构建，保持其核心模型架构，同时在训练和测试策略以及数据质量方面引入显著增强。本文深入探讨了模型规模与性能之间的关系，系统性地探索了视觉编码器、语言模型、数据集规模和测试时配置的性能趋势。通过在广泛的基准测试中进行大量评估，包括跨学科推理、文档理解、多图像/视频理解、真实世界理解、多模态幻觉检测、视觉定位、多语言能力和纯语言处理，InternVL 2.5 在性能上表现出竞争力，与 GPT-4o 和 Claude-3.5-Sonnet 等领先商业模型不相上下。值得注意的是，我们的模型是首个在 MMMU 基准测试上取得 70% 以上分数的开源 MLLM，通过链式思考（CoT）推理实现了 3.7 分的提升，展现出强大的测试时规模潜力。我们希望该模型为开源社区贡献，以制定开发和应用多模态 AI 系统的新标准。HuggingFace 演示见 https://huggingface.co/spaces/OpenGVLab/Intern
引用
@article{arxiv.2412.05271,
title = {Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling},
author = {Zhe Chen and Weiyun Wang and Yue Cao and Yangzhou Liu and Zhangwei Gao and Erfei Cui and Jinguo Zhu and Shenglong Ye and Hao Tian and Zhaoyang Liu and Lixin Gu and Xuehui Wang and Qingyun Li and Yiming Ren and Zixuan Chen and Jiapeng Luo and Jiahao Wang and Tan Jiang and Bo Wang and Conghui He and Botian Shi and Xingcheng Zhang and Han Lv and Yi Wang and Wenqi Shao and Pei Chu and Zhongying Tu and Tong He and Zhiyong Wu and Huipeng Deng and Jiaye Ge and Kai Chen and Kaipeng Zhang and Limin Wang and Min Dou and Lewei Lu and Xizhou Zhu and Tong Lu and Dahua Lin and Yu Qiao and Jifeng Dai and Wenhai Wang},
journal= {arXiv preprint arXiv:2412.05271},
year = {2025}
}
备注
Technical Report