The information loss of a stochastic map
Information Theory
2021-12-23 v3 Category Theory
math.IT
Probability
Abstract
We provide a stochastic extension of the Baez-Fritz-Leinster characterization of the Shannon information loss associated with a measure-preserving function. This recovers the conditional entropy and a closely related information-theoretic measure that we call conditional information loss. Although not functorial, these information measures are semi-functorial, a concept we introduce that is definable in any Markov category. We also introduce the notion of an entropic Bayes' rule for information measures, and we provide a characterization of conditional entropy in terms of this rule.
Keywords
Cite
@article{arxiv.2107.01975,
title = {The information loss of a stochastic map},
author = {James Fullwood and Arthur J. Parzygnat},
journal= {arXiv preprint arXiv:2107.01975},
year = {2021}
}
Comments
31 pages; Typos fixed in Defn 2.12 and the proofs of Prop 4.2 iii) and Prop 6.8 (numbering scheme differs from published version)