Related papers: Explicit Bounds for Entropy Concentration under Li…
The phenomenon of entropy concentration provides strong support for the maximum entropy method, MaxEnt, for inferring a probability vector from information in the form of constraints. Here we extend this phenomenon, in a discrete setting,…
The asymptotic behaviour of a closed BCMP network, with $n$ queues and $m_n$ clients, is analyzed when $n$ and $m_n$ become simultaneously large. Our method relies on Berry-Esseen type approximations coming in the Central Limit Theorem. We…
We derive novel and sharp high-dimensional Berry--Esseen bounds for the sum of $m$-dependent random vectors over the class of hyper-rectangles exhibiting only a poly-logarithmic dependence in the dimension. Our results hold under minimal…
In this work, we provide a $(n/m)^{-1/2}$-rate finite sample Berry-Esseen bound for $m$-dependent high-dimensional random vectors over the class of hyper-rectangles. This bound imposes minimal assumptions on the random vectors such as…
Berry-Esseen-type bounds for total variation and relative entropy distances to the normal law are established for the sums of non-i.i.d. random variables.
Concentration inequalities for the sample mean, like those due to Bernstein, Hoeffding, and Bentkus, are valid for any sample size but overly conservative, yielding confidence intervals that are unnecessarily wide. The central limit theorem…
Maximum entropy distributions with discrete support in $m$ dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions…
Relative entropy, as a divergence metric between two distributions, can be used for offline change-point detection and extends classical methods that mainly rely on moment-based discrepancies. To build a statistical test suitable for this…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
We give a characterization of Maximum Entropy/Minimum Relative Entropy inference by providing two `strong entropy concentration' theorems. These theorems unify and generalize Jaynes' `concentration phenomenon' and Van Campenhout and Cover's…
We establish a Berry--Esseen bound for general multivariate nonlinear statistics by developing a new multivariate-type randomized concentration inequality. The bound is the best possible for many known statistics. As applications,…
In this paper, we establish Berry--Esseen bounds for both self-normalized and non-self-normalized sums of locally dependent random variables. The proofs are based on Stein's method together with a concentration inequality approach. We…
Non-asymptotic bounds for Gaussian and bootstrap approximation have recently attracted significant interest in high-dimensional statistics. This paper studies Berry-Esseen bounds for such approximations with respect to the multivariate…
We investigate the theoretical foundations of a recently introduced entropy-based formulation of weighted least squares for the approximation of overdetermined linear systems, motivated by robust data fitting in the presence of sparse gross…
Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first…
For any integer $m<n$, where $m$ can depend on $n$, we study the rate of convergence of $\frac{1}{\sqrt{m}}\mathrm{Tr} \mathbf{U}^m$ to its limiting Gaussian as $n\to\infty$ for orthogonal, unitary and symplectic Haar distributed random…
In this work we study the rate of convergence in the central limit theorem for the Euclidean norm of random orthogonal projections of vectors chosen at random from an $\ell_p^n$-ball which has been obtained in [Alonso-Guti\'errez, Prochno,…
The Maximum Entropy Method (MEM) is a popular data analysis technique based on Bayesian inference, which has found various applications in the research literature. While the MEM itself is well-grounded in statistics, I argue that its…
We provide a Lyapunov type bound in the multivariate central limit theorem for sums of independent, but not necessarily identically distributed random vectors. The error in the normal approximation is estimated for certain classes of sets,…
This paper addresses the question of the fluctuations of the empirical entropy of a chain of infinite order. We assume that the chain takes values on a finite alphabet and loses memory exponentially fast. We consider two possible…