AI Uncertainty Quantification (UQ) has the potential to improve human decision-making beyond AI predictions alone by providing additional probabilistic information to users. The majority of past research on AI and human decision-making has concentrated on model explainability and interpretability, with little focus on understanding the potential impact of UQ on human decision-making. We evaluated the impact on human decision-making for instance-level UQ, calibrated using a strict scoring rule, in two online behavioral experiments. In the first experiment, our results showed that UQ was beneficial for decision-making performance compared to only AI predictions. In the second experiment, we found UQ had generalizable benefits for decision-making across a variety of representations for probabilistic information. These results indicate that implementing high quality, instance-level UQ for AI may improve decision-making with real systems compared to AI predictions alone.
@article{arxiv.2309.10852,
title = {Using AI Uncertainty Quantification to Improve Human Decision-Making},
author = {Laura R. Marusich and Jonathan Z. Bakdash and Yan Zhou and Murat Kantarcioglu},
journal= {arXiv preprint arXiv:2309.10852},
year = {2024}
}