Related papers: Finite-Sample Concentration of the Multinomial in …
This paper proposes and studies new quantum version of $f$-divergences, a class of convex functionals of a pair of probability distributions including Kullback-Leibler divergence, Rnyi-type relative entropy and so on. There are several…
Kullback--Leibler (KL) divergence is a fundamental measure of the dissimilarity between two probability distributions, but it can become unstable in high-dimensional settings due to its sensitivity to mismatches in distributional support.…
We consider the robust algorithms for the $k$-means clustering problem where a quantizer is constructed based on $N$ independent observations. Our main results are median of means based non-asymptotic excess distortion bounds that hold…
Let $G$ be a finite group generated by $k$ elements. The well-known product replacement algorithm provides an effective method for sampling generating sets of $G$. We study a refinement of this algorithm that is designed to output…
We provide finite-sample analysis of a general framework for using k-nearest neighbor statistics to estimate functionals of a nonparametric continuous probability density, including entropies and divergences. Rather than plugging a…
We present a new lower bound on the differential entropy rate of stationary processes whose sequences of probability density functions fulfill certain regularity conditions. This bound is obtained by showing that the gap between the…
Kullback-Leibler (KL) divergence is a fundamental concept in information theory that quantifies the discrepancy between two probability distributions. In the context of Variational Autoencoders (VAEs), it serves as a central regularization…
Experimental designs are tools which can dramatically reduce the number of simulations required by time-consuming computer codes. Because we don't know the true relation between the response and inputs, designs should allow one to fit a…
Estimates are constructed for the deviation of the concentration functions of sums of independent random variables with finite variances from the folded normal distribution function without any assumptions concerning the existence of the…
We consider the problem of estimating a rank-one perturbation of a Wigner matrix in a setting of low signal-to-noise ratio. This serves as a simple model for principal component analysis in high dimensions. The mutual information per…
We conduct a KL-divergence based procedure for testing elliptical distributions. The procedure simultaneously takes into account the two defining properties of an elliptically distributed random vector: independence between length and…
The evidence lower bound (ELBO) is one of the most central objectives for probabilistic unsupervised learning. For the ELBOs of several generative models and model classes, we here prove convergence to entropy sums. As one result, we…
We derive tight non-asymptotic bounds for the Kolmogorov distance between the probabilities of two Gaussian elements to hit a ball in a Hilbert space. The key property of these bounds is that they are dimension-free and depend on the…
Gradient-flow sampling interprets a Gibbs distribution as the minimizer of an energy functional over probability measures and generates dynamics converging to this target. Under spherical Hellinger-Kantorovich (SHK) geometry, the flow…
In this note we study sets of normal generators of finitely presented residually $p$-finite groups. We show that if an infinite, finitely presented, residually $p$-finite group $G$ is normally generated by $g_1,\dots,g_k$ with order…
Let $Q$ be a relatively compact subset in a Hilbert space $V$. For a given $\e>0$ let $N(\e,Q)$ be the minimal number of linear measurements, sufficient to reconstruct any $x \in Q$ with the accuracy $\e$. We call $N(\e,Q)$ a sampling…
In this paper, we derive some upper and lower bounds and inequalities for the total variation distance (TVD) and the Kullback-Leibler divergence (KLD), also known as the relative entropy, between two probability measures $\mu$ and $\nu$…
Recent years have seen a surge of interest in the algorithmic estimation of stochastic entropy production (EP) from trajectory data via machine learning. A crucial element of such algorithms is the identification of a loss function whose…
Our contribution is a bounded cubic compilation theorem. For each fixed resource parameter $k$, syntactic proof checking at resource level $k$ is faithfully represented by a finite bounded-domain system of cubic polynomial equations. Every…
We study the gradient flow for a relaxed approximation to the Kullback-Leibler (KL) divergence between a moving source and a fixed target distribution. This approximation, termed the KALE (KL approximate lower-bound estimator), solves a…