Existing benchmarks do not test Large Multimodal Models (LMMs) on their interactive intelligence with human users, which is vital for developing general-purpose AI assistants. We design InterFeedback, an interactive framework, which can be applied to any LMM and dataset to assess this ability autonomously. On top of this, we introduce InterFeedback-Bench which evaluates interactive intelligence using two representative datasets, MMMU-Pro and MathVerse, to test 10 different open-source LMMs. Additionally, we present InterFeedback-Human, a newly collected dataset of 120 cases designed for manually testing interactive performance in leading models such as OpenAI-o1 and Claude-Sonnet-4. Our evaluation results indicate that even the state-of-the-art LMM, OpenAI-o1, struggles to refine its responses based on human feedback, achieving an average score of less than 50%. Our findings point to the need for methods that can enhance LMMs' capabilities to interpret and benefit from feedback.
@article{arxiv.2502.15027,
title = {InterFeedback: Unveiling Interactive Intelligence of Large Multimodal Models via Human Feedback},
author = {Henry Hengyuan Zhao and Wenqi Pei and Yifei Tao and Haiyang Mei and Mike Zheng Shou},
journal= {arXiv preprint arXiv:2502.15027},
year = {2025}
}