Trusting Language Models in Education
Computation and Language
2023-08-09 v1 Artificial Intelligence
Abstract
Language Models are being widely used in Education. Even though modern deep learning models achieve very good performance on question-answering tasks, sometimes they make errors. To avoid misleading students by showing wrong answers, it is important to calibrate the confidence - that is, the prediction probability - of these models. In our work, we propose to use an XGBoost on top of BERT to output the corrected probabilities, using features based on the attention mechanism. Our hypothesis is that the level of uncertainty contained in the flow of attention is related to the quality of the model's response itself.
Cite
@article{arxiv.2308.03866,
title = {Trusting Language Models in Education},
author = {Jogi Suda Neto and Li Deng and Thejaswi Raya and Reza Shahbazi and Nick Liu and Adhitya Venkatesh and Miral Shah and Neeru Khosla and Rodrigo Capobianco Guido},
journal= {arXiv preprint arXiv:2308.03866},
year = {2023}
}