Related papers: Minimization Problems Based on Relative $\alpha$-E…
We study minimization of a parametric family of relative entropies, termed relative $\alpha$-entropies (denoted $\mathscr{I}_{\alpha}(P,Q)$). These arise as redundancies under mismatched compression when cumulants of compressed lengths are…
In part I of this two-part work, certain minimization problems based on a parametric family of relative entropies (denoted $\mathscr{I}_{\alpha}$) were studied. Such minimizers were called forward $\mathscr{I}_{\alpha}$-projections. Here, a…
This paper extends some geometric properties of a one-parameter family of relative entropies. These arise as redundancies when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative…
Entropy and relative or cross entropy measures are two very fundamental concepts in information theory and are also widely used for statistical inference across disciplines. The related optimization problems, in particular the maximization…
We prove characterization theorems for relative entropy (also known as Kullback-Leibler divergence), q-logarithmic entropy (also known as Tsallis entropy), and q-logarithmic relative entropy. All three have been characterized axiomatically…
Kullback-Leibler relative-entropy has unique properties in cases involving distributions resulting from relative-entropy minimization. Tsallis relative-entropy is a one parameter generalization of Kullback-Leibler relative-entropy in the…
We consider the problem of minimizing a generalized relative entropy, with respect to a reference diffusion law, over the set of path-measures with fully prescribed marginal distributions. When dealing with the actual relative entropy,…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
The asymptotic correspondence between the probability mass function of the $q$-deformed multinomial distribution and the $q$-generalised Kullback-Leibler divergence, also known as Tsallis relative entropy, is established. The probability…
Kullback-Leibler relative-entropy, in cases involving distributions resulting from relative-entropy minimization, has a celebrated property reminiscent of squared Euclidean distance: it satisfies an analogue of the Pythagoras' theorem. And…
Pinsker's and Fannes' type bounds on the Tsallis relative entropy are derived. The monotonicity property of the quantum $f$-divergence is used for its estimating from below. For order $\alpha\in(0,1)$, a family of lower bounds of Pinsker…
A comparative study of one-dimensional quantum structures which allow analytic expressions for the position and momentum R\'{e}nyi $R(\alpha)$ and Tsallis $T(\alpha)$ entropies, focuses on extracting the most characteristic physical…
A lower bound of the reduced relative entropy is given by the use of a variational expression. The reduced Tsallis relative entropy is defined and some results are given. In particular, the convexity of the reduced Tsallis relative entropy…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical…
This dissertation investigates relative entropies, also called generalized divergences, and how they can be used to characterize information-theoretic tasks in quantum information theory. The main goal is to further refine characterizations…
One way of getting insight into non-Gaussian measures, posed on infinite dimensional Hilbert spaces, is to first obtain best fit Gaussian approximations, which are more amenable to numerical approximation. These Gaussians can then be used…
In \cite{KumarS15J2}, it was shown that a generalized maximum likelihood estimation problem on a (canonical) $\alpha$-power-law model ($\mathbb{M}^{(\alpha)}$-family) can be solved by solving a system of linear equations. This was due to an…
We consider a two-parameter family of R\'enyi relative entropies $D_{\alpha,z}(\rho||\sigma)$ that are quantum generalisations of the classical R\'enyi divergence $D_{\alpha}(p||q)$. This family includes many known relative entropies (or…
Projection theorems of divergences enable us to find reverse projection of a divergence on a specific statistical model as a forward projection of the divergence on a different but rather "simpler" statistical model, which, in turn, results…