SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence
Abstract
We introduce SciEvalKit, a unified benchmarking toolkit designed to evaluate AI models for science across a broad range of scientific disciplines and task capabilities. Unlike general-purpose evaluation platforms, SciEvalKit focuses on the core competencies of scientific intelligence, including Scientific Multimodal Perception, Scientific Multimodal Reasoning, Scientific Multimodal Understanding, Scientific Symbolic Reasoning, Scientific Code Generation, Science Hypothesis Generation and Scientific Knowledge Understanding. It supports six major scientific domains, spanning from physics and chemistry to astronomy and materials science. SciEvalKit builds a foundation of expert-grade scientific benchmarks, curated from real-world, domain-specific datasets, ensuring that tasks reflect authentic scientific challenges. The toolkit features a flexible, extensible evaluation pipeline that enables batch evaluation across models and datasets, supports custom model and dataset integration, and provides transparent, reproducible, and comparable results. By bridging capability-based evaluation and disciplinary diversity, SciEvalKit offers a standardized yet customizable infrastructure to benchmark the next generation of scientific foundation models and intelligent agents. The toolkit is open-sourced and actively maintained to foster community-driven development and progress in AI4Science.
Cite
@article{arxiv.2512.22334,
title = {SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence},
author = {Yiheng Wang and Yixin Chen and Shuo Li and Yifan Zhou and Bo Liu and Hengjian Gao and Jiakang Yuan and Jia Bu and Wanghan Xu and Yuhao Zhou and Xiangyu Zhao and Zhiwang Zhou and Fengxiang Wang and Haodong Duan and Songyang Zhang and Jun Yao and Han Deng and Yizhou Wang and Jiabei Xiao and Jiaqi Liu and Encheng Su and Yujie Liu and Weida Wang and Junchi Yao and Shenghe Zheng and Haoran Sun and Runmin Ma and Xiangchao Yan and Bo Zhang and Dongzhan Zhou and Shufei Zhang and Peng Ye and Xiaosong Wang and Shixiang Tang and Wenlong Zhang and Lei Bai},
journal= {arXiv preprint arXiv:2512.22334},
year = {2026}
}