On the robustness of ChatGPT in teaching Korean Mathematics
Abstract
ChatGPT, an Artificial Intelligence model, has the potential to revolutionize education. However, its effectiveness in solving non-English questions remains uncertain. This study evaluates ChatGPT's robustness using 586 Korean mathematics questions. ChatGPT achieves 66.72% accuracy, correctly answering 391 out of 586 questions. We also assess its ability to rate mathematics questions based on eleven criteria and perform a topic analysis. Our findings show that ChatGPT's ratings align with educational theory and test-taker perspectives. While ChatGPT performs well in question classification, it struggles with non-English contexts, highlighting areas for improvement. Future research should address linguistic biases and enhance accuracy across diverse languages. Domain-specific optimizations and multilingual training could improve ChatGPT's role in personalized education.
Keywords
Cite
@article{arxiv.2502.11915,
title = {On the robustness of ChatGPT in teaching Korean Mathematics},
author = {Phuong-Nam Nguyen and Quang Nguyen-The and An Vu-Minh and Diep-Anh Nguyen and Xuan-Lam Pham},
journal= {arXiv preprint arXiv:2502.11915},
year = {2025}
}
Comments
21 pages, 12 figures, includes statistical analysis of ChatGPT's robustness in solving and rating multilingual mathematics questions. Focus on Korean CSAT Mathematics. Evaluates AI accuracy, rating effectiveness, and topic analysis