Related papers: Information Processing Equalities and the Informat…
This article expands the framework of Bayesian inference and provides direct probabilistic methods for approaching inference tasks that are typically handled with information theory. We treat Bayesian probability updating as a random…
Information theory provides tools to predict the performance of a learning algorithm on a given dataset. For instance, the accuracy of learning an unknown parameter can be upper bounded by reducing the learning task to hypothesis testing…
This paper deals with three major types of convergence of probability measures on metric spaces: weak convergence, setwise converges, and convergence in the total variation. First, it describes and compares necessary and sufficient…
We show that an information theoretic distance measured by the relative Fisher information between canonical equilibrium phase densities corresponding to forward and backward processes is intimately related to the gradient of the dissipated…
A broad class of informationally complete symmetric measurements is introduced. It can be understood as a common generalization of symmetric, informationally complete POVMs and mutually unbiased bases. Additionally, it provides a natural…
Within path sampling framework, we show that probability distribution divergences, such as the Chernoff information, can be estimated via thermodynamic integration. The Boltzmann-Gibbs distribution pertaining to different Hamiltonians is…
Information related measures are useful tools for multi variable data analysis, as measures of dependence among variables, and as descriptions of order in biological and physical systems. Information related measures, like marginal…
Estimation of the $\phi$-divergence between two unknown probability distributions using empirical data is a fundamental problem in information theory and statistical learning. We consider a multi-variate generalization of the data dependent…
In this article we present very intuitive, easy to follow, yet mathematically rigorous, approach to the so called data fitting process. Rather than minimizing the distance between measured and simulated data points, we prefer to find such…
Most metrics between finite point measures currently used in the literature have the flaw that they do not treat differing total masses in an adequate manner for applications. This paper introduces a new metric $\bar{d}_1$ that combines…
We consider the estimation of the transition matrix of a hidden Markovian process by using information geometry with respect to transition matrices. In this paper, only the histogram of $k$-memory data is used for the estimation. To…
There are many applications that benefit from computing the exact divergence between 2 discrete probability measures, including machine learning. Unfortunately, in the absence of any assumptions on the structure or independencies within…
We consider an open quantum system undergoing Markovian dynamics, the latter being modelled by a discrete-time quantum Markov semigroup $(\Phi^n)_{n \in {\mathbb{N}}}$, resulting from the action of sequential uses of a quantum channel…
There are three classical divergence measures known in the literature on information theory and statistics. These are namely, Jeffryes-Kullback-Leiber \cite{jef} \cite{kul} \textit{J-divergence}. Sibson-Burbea-Rao \cite{sib} \cite{bur1,…
Lifted probabilistic inference algorithms have been successfully applied to a large number of symmetric graphical models. Unfortunately, the majority of real-world graphical models is asymmetric. This is even the case for relational…
Information projections are the key building block of variational inference algorithms and are used to approximate a target probabilistic model by projecting it onto a family of tractable distributions. In general, there is no guarantee on…
Information geometry is a study of statistical manifolds, that is, spaces of probability distributions from a geometric perspective. Its classical information-theoretic applications relate to statistical concepts such as Fisher information,…
Meta-learning, or "learning to learn", refers to techniques that infer an inductive bias from data corresponding to multiple related tasks with the goal of improving the sample efficiency for new, previously unobserved, tasks. A key…
Markov Random Field models are powerful tools for the study of complex systems. However, little is known about how the interactions between the elements of such systems are encoded, especially from an information-theoretic perspective. In…
The importance of Fisher information is increasing in nonequilibrium thermodynamics, as it has played a fundamental role in trade-off relations such as thermodynamic uncertainty relations and speed limits. In this work, we investigate…