Related papers: Reweighted information inequalities
Two new information-theoretic methods are introduced for establishing Poisson approximation inequalities. First, using only elementary information-theoretic techniques it is shown that, when $S_n=\sum_{i=1}^nX_i$ is the sum of the (possibly…
Using martingale methods, we provide bounds for the entropy of a probability measure on $\mathbb {R}^d$ with the right-hand side given in a certain integral form. As a corollary, in the one-dimensional case, we obtain a weighted log-Sobolev…
Methods have been developed to identify the probability distribution of a random vector $Z$ from information consisting of its bounded range and the probability density function or moments of a quantity of interest, $Q(Z)$. The mapping from…
We develop a new framework for establishing approximate factorization of entropy on arbitrary probability spaces, using a geometric notion known as non-negative sectional curvature. The resulting estimates are equivalent to entropy…
We consider an inverse problem for the linear one-dimensional wave equation with variable coefficients consisting in determining an unknown source term from a boundary observation. A method to obtain approximations of this inverse problem…
An information theoretic measure is derived that quantifies the statistical coherence between systems evolving in time. The standard time delayed mutual information fails to distinguish information that is actually exchanged from shared…
Tight bounds for several symmetric divergence measures are introduced, given in terms of the total variation distance. Each of these bounds is attained by a pair of 2 or 3-element probability distributions. An application of these bounds…
The objective of Bayesian inference is often to infer, from data, a probability measure for a random variable that can be used as input for Monte Carlo simulation. When datasets for Bayesian inference are small, a principle challenge is…
We find sufficient conditions for a probability measure $\mu$ to satisfy an inequality of the type $$ \int_{\R^d} f^2 F\Bigl(\frac{f^2}{\int_{\R^d} f^2 d \mu} \Bigr) d \mu \le C \int_{\R^d} f^2 c^{*}\Bigl(\frac{|\nabla f|}{|f|} \Bigr) d \mu…
In this paper, we give necessary and sufficient conditions for Talagrand's like transportation cost inequalities on the real line. This brings a new wide class of examples of probability measures enjoying a dimension-free concentration of…
We provide a stochastic extension of the Baez-Fritz-Leinster characterization of the Shannon information loss associated with a measure-preserving function. This recovers the conditional entropy and a closely related information-theoretic…
In this paper, we first introduce and define several new information divergences in the space of transition matrices of finite Markov chains which measure the discrepancy between two Markov chains. These divergences offer natural…
Mixture distributions are a workhorse model for multimodal data in information theory, signal processing, and machine learning. Yet even when each component density is simple, the differential entropy of the mixture is notoriously hard to…
We develop a general method to study the Fisher information distance in central limit theorem for nonlinear statistics. We first construct completely new representations for the score function. We then use these representations to derive…
We prove a log-Sobolev inequality for a certain class of log-concave measures in high dimension. These are the probability measures supported on the unit cube in R^n whose density takes the form exp(-H) where the function H is assumed to be…
We prove a version of Talagrand's concentration inequality for subordinated sub-Laplacian on a compact Riemannian manifold using tools from noncommutative geometry. As an application, motivated by quantum information theory, we show that on…
We introduce a framework for obtaining tight mixing times for Markov chains based on what we call restricted modified log-Sobolev inequalities. Modified log-Sobolev inequalities (MLSI) quantify the rate of relative entropy contraction for…
Statistical inference more often than not involves models which are non-linear in the parameters thus leading to non-Gaussian posteriors. Many computational and analytical tools exist that can deal with non-Gaussian distributions, and…
Obtaining meaningful quantitative descriptions of the statistical dependence within multivariate systems is a difficult open problem. Recently, the Partial Information Decomposition (PID) was proposed to decompose mutual information (MI)…
The two principal/immediate influences -- which we seek to interrelate here -- upon the undertaking of this study are papers of Zyczkowski and Slomczy\'nski (J. Phys. A 34, 6689 [2001]) and of Petz and Sudar (J. Math. Phys. 37, 2262…