Related papers: A Bayesian Characterization of Relative Entropy
In Bayesian statistics probability distributions express beliefs. However, for many problems the beliefs cannot be computed analytically and approximations of beliefs are needed. We seek a loss function that quantifies how "embarrassing" it…
Finding the minimal relative entropy of two quantum states under semidefinite constraints is a pivotal problem located at the mathematical core of various applications in quantum information theory. An efficient method for providing…
The measure-theoretic definition of Kullback-Leibler relative-entropy (KL-entropy) plays a basic role in the definitions of classical information measures. Entropy, mutual information and conditional forms of entropy can be expressed in…
Heisenberg's uncertainty principle has recently led to general measurement uncertainty relations for quantum systems: incompatible observables can be measured jointly or in sequence only with some unavoidable approximation, which can be…
This paper develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…
This thesis develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…
Consanguinity of entropy and complexity is pointed out through the example of coherent states of the group $SL(d+1,\C)$. Both are obtained from the K\"ahler potential of the underlying geometry of the sphere corresponding to the…
We show that the Schnakenberg's entropy production rate in a master equation is lower bounded by a function of the weight of the Markov graph, here defined as the sum of the absolute values of probability currents over the edges. The result…
New upper bounds on the relative entropy are derived as a function of the total variation distance. One bound refines an inequality by Verd\'{u} for general probability measures. A second bound improves the tightness of an inequality by…
We study optimal solutions to an abstract optimization problem for measures, which is a generalization of classical variational problems in information theory and statistical physics. In the classical problems, information and relative…
This work presents an upper-bound to value that the Kullback-Leibler (KL) divergence can reach for a class of probability distributions called quantum distributions (QD). The aim is to find a distribution $U$ which maximizes the KL…
Observational entropy -- a quantity that unifies Boltzmann's entropy, Gibbs' entropy, von Neumann's macroscopic entropy, and the diagonal entropy -- has recently been argued to play a key role in a modern formulation of statistical…
A loss function measures the discrepancy between the true values (observations) and their estimated fits, for a given instance of data. A loss function is said to be proper (unbiased, Fisher consistent) if the fits are defined over a unit…
In this paper we study algorithms to find a Gaussian approximation to a target measure defined on a Hilbert space of functions; the target measure itself is defined via its density with respect to a reference Gaussian measure. We employ the…
A two parameter generalization of Boltzmann-Gibbs-Shannon entropy based on natural logarithm is introduced. The generalization of the Shannon-Kinchinn axioms corresponding to the two parameter entropy is proposed and verified. We present…
The characteristic function of the folded normal distribution and its moment function are derived. The entropy of the folded normal distribution and the Kullback--Leibler from the normal and half normal distributions are approximated using…
A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is…
In this paper it was proved that the quantum relative entropy $D(\sigma \| \rho)$ can be asymptotically attained by Kullback Leibler divergences of probabilities given by a certain sequence of POVMs. The sequence of POVMs depends on $\rho$,…
Many thermodynamic relations involve inequalities, with equality if a process does not involve dissipation. In this article we provide equalities in which the dissipative contribution is shown to involve the relative entropy (a.k.a.…
Information theory on a time-discrete setting in the framework of time series analysis is generalized to the time-continuous case. Considerations of the Roessler and Lorenz dynamics as well as the Ornstein-Uhlenbeck process yield for…