Related papers: Finite-Sample Concentration of the Multinomial in …
A well-known technique in estimating probabilities of rare events in general and in information theory in particular (used, e.g., in the sphere-packing bound), is that of finding a reference probability measure under which the event of…
We investigate the limiting behavior of sample central moments, examining the special cases where the limiting (as the sample size tends to infinity) distribution is degenerate. Parent (non-degenerate) distributions with this property are…
Estimating the Kullback-Leibler (KL) divergence between two distributions given samples from them is well-studied in machine learning and information theory. Motivated by considerations of multi-group fairness, we seek KL divergence…
Data collection is a critical step in statistical inference and data science, and the goal of statistical experimental design (ED) is to find the data collection setup that can provide most information for the inference. In this work we…
This work explores connections between the quantum relative entropy of two faithful states $\rho,\sigma$ (i.e. full-rank density matrices) and the Kullback-Leibler divergences of classical measures $\mu,\nu$. Here, $\mu$ and $\nu$ are…
A classic definition of multisensory integration (MI) has been proposed as ``the presence of a (statistically) significant change in the response to a cross-modal stimulus complex compared to unimodal stimuli''. However, this general…
For a random vector X in R^n, we obtain bounds on the size of a sample, for which the empirical p-th moments of linear functionals are close to the exact ones uniformly on an n-dimensional convex body K. We prove an estimate for a general…
We present novel and sharp lower bounds for higher load moments in the classical problem of mapping $M$ balls into $N$ bins by $q$-universal hashing, specialized to the case when $M=N$. As a corollary we prove a tight counterpart for the…
We study the problem of nonparametric estimation of density functions with a product form on the domain $\triangle=\{( x_1, \ldots, x_d)\in \mathbb{R}^d, 0\leq x_1\leq \dots \leq x_d \leq 1\}$. Such densities appear in the random truncation…
Discretization of continuous-time diffusion processes is a widely recognized method for sampling. However, it seems to be a considerable restriction when the potentials are often required to be smooth (gradient Lipschitz). This paper…
We consider the problem of estimating probability density functions based on sample data, using a finite mixture of densities from some component class. To this end, we introduce the $h$-lifted Kullback--Leibler (KL) divergence as a…
The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…
The main result of the paper is the following. Let a non-degenerate distribution have finite moments $\mu_k$ of all orders $k=0,1,2,\ldots$. Then the sequence $\{\mu_k/k!, \; k=0,1,2,\ldots\}$ either contains infinitely many different terms…
Generative models have achieved remarkable success across a range of applications, yet their evaluation still lacks principled uncertainty quantification. In this paper, we develop a method for comparing how close different generative…
In this paper it was proved that the quantum relative entropy $D(\sigma \| \rho)$ can be asymptotically attained by Kullback Leibler divergences of probabilities given by a certain sequence of POVMs. The sequence of POVMs depends on $\rho$,…
Testing whether two multivariate samples exhibit the same extremal behavior is an important problem in various fields including environmental and climate sciences. While several ad-hoc approaches exist in the literature, they often lack…
Consider the problem of estimating the Shannon entropy of a distribution over $k$ elements from $n$ independent samples. We show that the minimax mean-square error is within universal multiplicative constant factors of $$\Big(\frac{k }{n…
We obtain dimension-free concentration inequalities for $\ell^p$-norms, $p\geq2$, of infinitely divisible random vectors with independent coordinates and finite exponential moments. Besides such norms, the methods and results extend to some…
The irreversibility of a stationary time series can be quantified using the Kullback-Leibler divergence (KLD) between the probability to observe the series and the probability to observe the time-reversed series. Moreover, this KLD is a…
Training generative models to sample from unnormalized density functions is an important and challenging task in machine learning. Traditional training methods often rely on the reverse Kullback-Leibler (KL) divergence due to its…