Related papers: On a 2-relative entropy

We give a categorical treatment, in the spirit of Baez and Fritz, of relative entropy for probability distributions defined on standard Borel spaces. We define a category suitable for reasoning about statistical inference on standard Borel…

We give a new characterization of relative entropy, also known as the Kullback-Leibler divergence. We use a number of interesting categories related to probability theory. In particular, we consider a category FinStat where an object is a…

A Bayesian functorial characterization of the classical relative entropy (KL divergence) of finite probabilities was recently obtained by Baez and Fritz. This was then generalized to standard Borel spaces by Gagn\'e and Panangaden. Here, we…

Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…

Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical…

We develop information-theoretic measures of spatial structure and pattern in more than one dimension. As is well known, the entropy density of a two-dimensional configuration can be efficiently and accurately estimated via a converging…

In this work, we introduce a 2-categorical variant of Lurie's relative nerve functor. We prove that it defines a right Quillen equivalence which, upon passage to $\infty$-categorical localizations, corresponds to Lurie's scaled…

We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…

In analogy to the topological entropy for continuous endomorphisms of totally disconnected locally compact groups, we introduce a notion of topological entropy for continuous endomorphisms of locally linearly compact vector spaces. We study…

The present paper studies continuity of generalized entropy functions and relative entropies defined using the notion of a deformed logarithmic function. In particular, two distinct definitions of relative entropy are discussed. As an…

Various properties of relative entropy have led to its widespread use in information theory. These properties suggest that relative entropy has a role to play in systems that attempt to perform inference in terms of probability…

We introduce a new class of (not necessarily convex) bodies and show, among other things, that these bodies provide yet another link between convex geometric analysis and information theory. Namely, they give geometric interpretations of…

We study the relation of relative topological entropy and relative mean dimension between a factor map and its induced factor map for amenable group actions. On the one hand, we prove that a factor map has zero relative topological entropy…

We characterize a number of well known systems of approximate inference as loss models: lax sections of 2-fibrations of statistical games, constructed by attaching internally-defined loss functions to Bayesian lenses. Our examples include…

A two parameter generalization of Boltzmann-Gibbs-Shannon entropy based on natural logarithm is introduced. The generalization of the Shannon-Kinchinn axioms corresponding to the two parameter entropy is proposed and verified. We present…

By using Araki's relative entropy, Lieb's convexity and the theory of singular integrals, we compute the mutual information associated with free fermions, and we deduce many results about entropies for chiral CFT's which are embedded into…

Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…

Let $A$ and $B$ be two accretive operators. We first introduce the weighted geometric mean of $A$ and $B$ together with some related properties. Afterwards, we define the relative entropy as well as the Tsallis entropy of $A$ and $B$. The…

Relative entropy is a measure of distinguishability for quantum states, and plays a central role in quantum information theory. The family of Renyi entropies generalizes to Renyi relative entropies that include as special cases most entropy…

We introduce a new generalization of relative entropy to non-negative vectors with sums $\gt 1$. We show in a purely combinatorial setting, with no probabilistic considerations, that in the presence of linear constraints defining a convex…