Related papers: Sharp Concentration Inequalities for the Centered …
We study the question of closeness testing for two discrete distributions. More precisely, given samples from two distributions $p$ and $q$ over an $n$-element set, we wish to distinguish whether $p=q$ versus $p$ is at least $\eps$-far from…
We prove anti-concentration bounds for the inner product of two independent random vectors, and use these bounds to prove lower bounds in communication complexity. We show that if $A,B$ are subsets of the cube $\{\pm 1\}^n$ with $|A| \cdot…
We present some new nonparametric estimators of entropies and we establish almost sure consistency and central limit Theorems for some of the most important entropies in the discrete case. Our theorical results are validated by simulations.
Existing training criteria in automatic speech recognition(ASR) permit the model to freely explore more than one time alignments between the feature and label sequences. In this paper, we use entropy to measure a model's uncertainty, i.e.…
In many problems in data mining and machine learning, data items that need to be clustered or classified are not points in a high-dimensional space, but are distributions (points on a high dimensional simplex). For distributions, natural…
We consider the problem of estimating functionals of discrete distributions, and focus on tight nonasymptotic analysis of the worst case squared error risk of widely used estimators. We apply concentration inequalities to analyze the random…
We study concentration inequalities for the Kullback--Leibler (KL) divergence between the empirical distribution and the true distribution. Applying a recursion technique, we improve over the method of types bound uniformly in all regimes…
For probability measures on countable spaces we derive distributional limits for empirical entropic optimal transport quantities. More precisely, we show that the empirical optimal transport plan weakly converges to a centered Gaussian…
We propose an information-theoretical measure, the \textit{relative cluster entropy} $\mathcal{D_{C}}[P \| Q] $, to discriminate among cluster partitions characterised by probability distribution functions $P$ and $Q$. The measure is…
In this work we present concentration inequalities for the sum $S_n$ of independent integer-valued not necessary indentically distributed random variables, where each variable has tail function that can be bounded by some power function…
For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one…
We study the relative entropy and the trace square distance, both of which measure the distance between reduced density matrices of two excited states in two dimensional conformal field theories. We find a general formula for the relative…
We establish a lower bound on the entropy of weighted sums of (possibly dependent) random variables $(X_1, X_2, \dots, X_n)$ possessing a symmetric joint distribution. Our lower bound is in terms of the joint entropy of $(X_1, X_2, \dots,…
Shannon entropy is often a quantity of interest to linguists studying the communicative capacity of human language. However, entropy must typically be estimated from observed data because researchers do not have access to the underlying…
We study the exact constants in the moment inequalities for sums of centered independent random variables: improve their asymptotics, low and upper bounds, calculate more exact asymptotics, elaborate the numerical algorithm for their…
We investigate concentration properties of functions of random vectors with values in the discrete cube, satisfying the stochastic covering property (SCP) or the strong Rayleigh property (SRP). Our result for SCP measures include…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…
We investigate the limiting behavior of sample central moments, examining the special cases where the limiting (as the sample size tends to infinity) distribution is degenerate. Parent (non-degenerate) distributions with this property are…
Given a probability distribution P, what is the minimum amount of bits needed to store a value x sampled according to P, such that x can later be recovered (except with some small probability)? Or, what is the maximum amount of uniform…
The principle of maximum entropy provides a useful method for inferring statistical mechanics models from observations in correlated systems, and is widely used in a variety of fields where accurate data are available. While the assumptions…