Related papers: Finite-Sample Concentration of the Multinomial in …
This paper deals with maximization of classical $f$-divergence between the distributions of a measurement outputs of a given pair of quantum states. $f$-divergence $D_{f}$ between the probability density functions $p_{1}$ and $p_{2}$ over a…
We prove a Bernstein-type bound for the difference between the average of negative log-likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the…
In this paper, we study a matricial version of the Byrnes-Georgiou-Lindquist generalized moment problem with complexity constraint. We introduce a new metric on multivariable spectral densities induced by the family of their spectral…
A finite sampling theory associated with a unitary representation of a finite non Abelian group $\mathbf{G}$ on a Hilbert space is stablished. The non Abelian group $\mathbf{G}$ is a knit product $\mathbf{N}\bowtie \mathbf{H}$ of two finite…
We propose a deep generative approach to sampling from a conditional distribution based on a unified formulation of conditional distribution and generalized nonparametric regression function using the noise-outsourcing lemma. The proposed…
We embark on a systematic study of the $(k+1)$-th derivative of $x^{k-r}H(x^r)$, where $H(x):=-x\log x-(1-x)\log(1-x)$ is the binary entropy and $k>r\geq 1$ are integers. Our motivation is the conjectural entropy inequality $\alpha_k…
We study the existence of the (thermodynamic) limit of the scaled cumulant-generating function L_n(z)=|W_n|^{-1}\logE\exp{z|\Xi\cap W_n|} of the empirical volume fraction |\Xi\cap W_n|/|W_n|, where |\cdot| denotes the d-dimensional Lebesgue…
The rate function for large deviations of the finite time Lyapunov exponent for the derived process in TM corresponding to a stochastic differential equation in M is related, via the Gartner-Ellis theorem, to the p-th moment Lyapunov…
The Pinsker inequality lower bounds the Kullback--Leibler divergence $D_{\textrm{KL}}$ in terms of total variation and provides a canonical way to convert $D_{\textrm{KL}}$ control into $\lVert \cdot \rVert_1$-control. Motivated by…
Mixture distributions are a workhorse model for multimodal data in information theory, signal processing, and machine learning. Yet even when each component density is simple, the differential entropy of the mixture is notoriously hard to…
Maximum entropy (MAXENT) method has a large number of applications in theoretical and applied machine learning, since it provides a convenient non-parametric tool for estimating unknown probabilities. The method is a major contribution of…
Traditionally, the Method of (Shannon-Kullback's) Relative Entropy Maximization (REM) is considered with linear moment constraints. In this work, the method is studied under frequency moment constraints which are non-linear in…
The second law of thermodynamics governs that nonequilibrium systems evolve towards states of higher entropy over time. However, it does not specify the rate of this evolution and the role of fluctuations that impact the system's dynamics.…
For simplest colored branching processes we prove an analog to the McMillan theorem and calculate Hausdorff dimensions of random fractals defined in terms of the limit behavior of empirical measures generated by finite genetic lines. In…
The purpose of the present paper is to establish moment estimates of Rosenthal type for a rather general class of random variables satisfying certain bounds on the cumulants. We consider sequences of random variables which satisfy a central…
The efficiency of Monte Carlo samplers is dictated not only by energetic effects, such as large barriers, but also by entropic effects that are due to the sheer volume that is sampled. The latter effects appear in the form of an entropic…
This archiving article consists of several short reports on the discussions between the two authors over the past two years at Oxford and Madrid, and their work carried out during that period on the upper bound of the Kullback-Leibler…
Let the term $k$-representation refer to the permutation representations of the symmetric group $\mathfrak{S}_n$ on $k$-tuples and $k$-subsets as well as the $S^{(n-k,1^k)}$ irreducible representation of $\mathfrak{S}_n$. Endow…
R\'enyi divergence is related to R\'enyi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and comes up in many settings. It was introduced by R\'enyi as…
This paper provides a unified perspective for the Kullback-Leibler (KL)-divergence and the integral probability metrics (IPMs) from the perspective of maximum likelihood density-ratio estimation (DRE). Both the KL-divergence and the IPMs…