相关论文: Sup-sums principles for F-divergence, Kullback--Le…
This paper proposes a new family of lower and upper bounds on the minimum mean squared error (MMSE). The key idea is to minimize/maximize the MMSE subject to the constraint that the joint distribution of the input-output statistics lies in…
In the context of averaging an inhomogeneous cosmological model, we propose a natural measure identical to the Kullback-Leibler relative information entropy, which expresses the distinguishability of the local inhomogeneous density field…
This paper develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…
A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is…
The LogSumExp function, dual to the Kullback-Leibler (KL) divergence, plays a central role in many important optimization problems, including entropy-regularized optimal transport (OT) and distributionally robust optimization (DRO). In…
Information generating functions have been used for generating various entropy and divergence measures. In the present work, we introduce quantile based relative information generating function and study its properties. The proposed…
A generalized Kullback-Leibler relative entropy is introduced starting with the symmetric Jackson derivative of the generalized overlap between two probability distributions. The generalization retains much of the structure possessed by the…
The asymptotic correspondence between the probability mass function of the $q$-deformed multinomial distribution and the $q$-generalised Kullback-Leibler divergence, also known as Tsallis relative entropy, is established. The probability…
In this paper we review various information-theoretic characterizations of the approach to equilibrium in biological systems. The replicator equation, evolutionary game theory, Markov processes and chemical reaction networks all describe…
Extensions of Fannes' inequality with partial sums of the Tsallis entropy are obtained for both the classical and quantum cases. The definition of kth partial sum under the prescribed order of terms is given. Basic properties of introduced…
We develop a fine-scale local analysis of measure entropy and measure sequence entropy based on combinatorial independence. The concepts of measure IE-tuples and measure IN-tuples are introduced and studied in analogy with their…
Blasiok (SODA'18) recently introduced the notion of a subgaussian sampler, defined as an averaging sampler for approximating the mean of functions $f:\{0,1\}^m \to \mathbb{R}$ such that $f(U_m)$ has subgaussian tails, and asked for explicit…
The Pinsker inequality lower bounds the Kullback--Leibler divergence $D_{\textrm{KL}}$ in terms of total variation and provides a canonical way to convert $D_{\textrm{KL}}$ control into $\lVert \cdot \rVert_1$-control. Motivated by…
In view of the minimization of a function which is the sum of a differentiable function $f$ and a convex function $g$ we introduce descent methods which can be viewed as produced by inexact auxiliary problem principleor inexact variable…
We consider M-estimators and derive supremal-inequalities of exponential-or polynomial type according as a boundedness- or a moment-condition is fulfilled. This enables us to derive rates of r-complete convergence and also to show r-qick…
This note describes Fatou's lemma and Lebesgue's dominated convergence theorem for a sequence of measures converging weakly to a finite measure and for a sequence of functions whose negative parts are uniformly integrable with respect to…
This paper mainly studies nonnegativity decision of forms based on variable substitutions. Unlike existing research, the paper regards simplex subdivisions as new perspectives to study variable substitutions, gives some subdivisions of the…
We consider finite-dimensional linear Gaussian Bayesian inverse problems with uncorrelated sensor measurements. In this setting, it is known that the expected information gain, quantified by the expected Kullback-Leibler divergence from the…
We adapt the concept of $\mathcal{K}-$convergence of Young measures to the sequences of approximate solutions resulting from numerical schemes. We obtain new results on pointwise convergence of numerical solutions in the case when solutions…
This paper extends the asymmetric Kullback-Leibler divergence and symmetric Jensen-Shannon divergence from two probability measures to the case of two sets of probability measures. We establish some fundamental properties of these…