This paper presents the Korean National Educational Test Benchmark (KoNET), a new benchmark designed to evaluate Multimodal Generative AI Systems using Korean national educational tests. KoNET comprises four exams: the Korean Elementary General Educational Development Test (KoEGED), Middle (KoMGED), High (KoHGED), and College Scholastic Ability Test (KoCSAT). These exams are renowned for their rigorous standards and diverse questions, facilitating a comprehensive analysis of AI performance across different educational levels. By focusing on Korean, KoNET provides insights into model performance in less-explored languages. We assess a range of models - open-source, open-access, and closed APIs - by examining difficulties, subject diversity, and human error rates. The code and dataset builder will be made fully open-sourced at https://github.com/naver-ai/KoNET.
Cite
@article{arxiv.2502.15422,
title = {Evaluating Multimodal Generative AI with Korean Educational Standards},
author = {Sanghee Park and Geewook Kim},
journal= {arXiv preprint arXiv:2502.15422},
year = {2025}
}
Comments
18 pages; To appear at NAACL 2025 Main Conference (Project page: https://github.com/naver-ai/KoNET )