Quantifying Calibration Error in Neural Networks Through Evidence-Based Theory
Abstract
Trustworthiness in neural networks is crucial for their deployment in critical applications, where reliability, confidence, and uncertainty play pivotal roles in decision-making. Traditional performance metrics such as accuracy and precision fail to capture these aspects, particularly in cases where models exhibit overconfidence. To address these limitations, this paper introduces a novel framework for quantifying the trustworthiness of neural networks by incorporating subjective logic into the evaluation of Expected Calibration Error (ECE). This method provides a comprehensive measure of trust, disbelief, and uncertainty by clustering predicted probabilities and fusing opinions using appropriate fusion operators. We demonstrate the effectiveness of this approach through experiments on MNIST and CIFAR-10 datasets, where post-calibration results indicate improved trustworthiness. The proposed framework offers a more interpretable and nuanced assessment of AI models, with potential applications in sensitive domains such as healthcare and autonomous systems.
Cite
@article{arxiv.2411.00265,
title = {Quantifying Calibration Error in Neural Networks Through Evidence-Based Theory},
author = {Koffi Ismael Ouattara and Ioannis Krontiris and Theo Dimitrakos and Frank Kargl},
journal= {arXiv preprint arXiv:2411.00265},
year = {2025}
}
Comments
This is the preprint of the paper accepted to Fusion 2025 (28th International Conference on Information Fusion, Rio de Janeiro, Brazil, July 7-10, 2025). The published version is available at https://doi.org/10.23919/FUSION65864.2025.11124121