不确定性量化的速率失真视角
机器学习
2024-06-19 v2 人工智能
机器学习
摘要
在监督学习中，了解输入与训练数据之间的接近程度有助于模型决定是否具有足够的证据来做出可靠预测。虽然像高斯过程这类强大的概率模型天然具备这一属性，但深度神经网络往往缺乏这一属性。在本文中，我们引入了Distance Aware Bottleneck(DAB)，即一种为深度神经网络增强这一属性的新方法。基于先前的信息瓶颈方法，我们的方法学习一个存储训练期间所有输入压缩表示的码本。新示例与该码本的距离可作为该示例的不确定性估计。 resulting model is simple to train and provides deterministic uncertainty estimates by a single forward pass. Finally, our method achieves better out-of-distribution (OOD) detection and misclassification prediction than prior methods, including expensive ensemble methods, deep kernel Gaussian Processes, and approaches based on the standard information bottleneck.
引用
@article{arxiv.2406.10775,
title = {A Rate-Distortion View of Uncertainty Quantification},
author = {Ifigeneia Apostolopoulou and Benjamin Eysenbach and Frank Nielsen and Artur Dubrawski},
journal= {arXiv preprint arXiv:2406.10775},
year = {2024}
}