Conformal Prediction with Large Language Models for Multi-Choice Question Answering
Abstract
As large language models continue to be widely developed, robust uncertainty quantification techniques will become crucial for their safe deployment in high-stakes scenarios. In this work, we explore how conformal prediction can be used to provide uncertainty quantification in language models for the specific task of multiple-choice question-answering. We find that the uncertainty estimates from conformal prediction are tightly correlated with prediction accuracy. This observation can be useful for downstream applications such as selective classification and filtering out low-quality predictions. We also investigate the exchangeability assumption required by conformal prediction to out-of-subject questions, which may be a more realistic scenario for many practical applications. Our work contributes towards more trustworthy and reliable usage of large language models in safety-critical situations, where robust guarantees of error rate are required.
Cite
@article{arxiv.2305.18404,
title = {Conformal Prediction with Large Language Models for Multi-Choice Question Answering},
author = {Bhawesh Kumar and Charlie Lu and Gauri Gupta and Anil Palepu and David Bellamy and Ramesh Raskar and Andrew Beam},
journal= {arXiv preprint arXiv:2305.18404},
year = {2023}
}
Comments
Updated sections on prompt engineering. Expanded sections 4.1 and 4.2 and appendix. Included additional references. Work published at the ICML 2023 (Neural Conversational AI TEACH) workshop