Multimodal Large Language Models (MLLMs) have demonstrated significant potential to advance a broad range of domains. However, current benchmarks for evaluating MLLMs primarily emphasize general knowledge and vertical step-by-step reasoning typical of STEM disciplines, while overlooking the distinct needs and potential of the Humanities and Social Sciences (HSS). Tasks in the HSS domain require more horizontal, interdisciplinary thinking and a deep integration of knowledge across related fields, which presents unique challenges for MLLMs, particularly in linking abstract concepts with corresponding visual representations. Addressing this gap, we present HSSBench, a dedicated benchmark designed to assess the capabilities of MLLMs on HSS tasks in multiple languages, including the six official languages of the United Nations. We also introduce a novel data generation pipeline tailored for HSS scenarios, in which multiple domain experts and automated agents collaborate to generate and iteratively refine each sample. HSSBench contains over 13,000 meticulously designed samples, covering six key categories. We benchmark more than 20 mainstream MLLMs on HSSBench and demonstrate that it poses significant challenges even for state-of-the-art models. We hope that this benchmark will inspire further research into enhancing the cross-disciplinary reasoning abilities of MLLMs, especially their capacity to internalize and connect knowledge across fields.
@article{arxiv.2506.03922,
title = {HSSBench: Benchmarking Humanities and Social Sciences Ability for Multimodal Large Language Models},
author = {Zhaolu Kang and Junhao Gong and Jiaxu Yan and Wanke Xia and Yian Wang and Ziwen Wang and Huaxuan Ding and Zhuo Cheng and Wenhao Cao and Zhiyuan Feng and Siqi He and Shannan Yan and Junzhe Chen and Xiaomin He and Chaoya Jiang and Wei Ye and Kaidong Yu and Xuelong Li},
journal= {arXiv preprint arXiv:2506.03922},
year = {2026}
}