English

How to quantify the coherence of a set of beliefs

Probability 2024-12-05 v1

Abstract

Given conflicting probability estimates for a set of events, how can we quantify how much they conflict? How can we find a single probability distribution that best encapsulates the given estimates? One approach is to minimize a loss function such as binary KL-divergence that quantifies the dissimilarity between the given estimates and the candidate probability distribution. Given a set of events, we characterize the facets of the polytope of coherent probability estimates about those events. We explore two applications of these ideas: eliciting the beliefs of large language models, and merging expert forecasts into a single coherent forecast.

Keywords

Cite

@article{arxiv.2412.02777,
  title  = {How to quantify the coherence of a set of beliefs},
  author = {Rowan Hess and Lionel Levine},
  journal= {arXiv preprint arXiv:2412.02777},
  year   = {2024}
}

Comments

33 pages, 3 figures, and 4 tables