Related papers: An extended Generalised Variance, with Application…
For a random variable we can define a variational relationship with practical physical meaning as dI=dbar(x)-bar(dx), where I is called as uncertainty measurement. With the help of a generalized definition of expectation,…
We study the problem of learning mixtures of $k$ Gaussians in $d$ dimensions. We make no separation assumptions on the underlying mixture components: we only require that the covariance matrices have bounded condition number and that the…
We present a new perspective of assessing the rates of convergence to the Gaussian and Poisson distributions in the Erd\"os-Kac theorem for additive arithmetic functions $\psi$ of a random integer $J_n$ uniformly distributed over…
Differential entropy and log determinant of the covariance matrix of a multivariate Gaussian distribution have many applications in coding, communications, signal processing and statistical inference. In this paper we consider in the high…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
Let $\mu$ be the equilibrium measure of an endomorphism of ${\sf P}^k({\bf C})$. We show that it is its unique measure of maximal entropy. We build $\mu$ directly as the distribution of any point outside an algebraic exceptional set.
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
We propose a nonparametric estimator of multivariate joint entropy based on partitioned sample spacing (PSS). The method extends univariate spacing ideas to $\mathbb{R}^{d}$ by partitioning into localized cells and aggregating within-cell…
The time variation of entropy, as an alternative to the variance, is proposed as a measure of the diffusion rate. It is shown that for linear and time-translationally invariant systems having a large-time limit for the density, at large…
In this paper, we obtain a new characterization result for symmetric distributions based on the entropy measure. Using the characterization, we propose a nonparametric test to test the symmetry of a distribution. We also develop the…
In this note, we show that the relative entropy of an empirical distribution of $n$ samples drawn from a set of size $k$ with respect to the true underlying distribution is exponentially concentrated around its expectation, with central…
We show how to determine the maximum and minimum possible values of one measure of entropy for a given value of another measure of entropy. These maximum and minimum values are obtained for two standard forms of probability distribution (or…
We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, R\'enyi, generalized R\'enyi, Tsallis, Sharma-Mittal, and modified Shannon entropy,…
Many authors have studied the phenomenon of typically Gaussian marginals of high-dimensional random vectors; e.g., for a probability measure on $\R^d$, under mild conditions, most one-dimensional marginals are approximately Gaussian if $d$…
Given n observations, we study the consistency of a batch of k new observations, in terms of their distribution function. We propose a non-parametric, non-likelihood test based on Edgeworth expansion of the distribution function. The…
We study variational principles for metric mean dimension. First we prove that in the variational principle of Lindenstrauss and Tsukamoto it suffices to take supremum over ergodic measures. Second we derive a variational principle for…
Consider a sphere of radius root(n) in n dimensions, and consider X, a random variable uniformly distributed on its surface. Poincare's Observation states that for large n, the distribution of the first k coordinates of X is close in total…
We introduce and initiate the study of new parameters associated with any norm and any log-concave measure on $\mathbb R^n$, which provide sharp distributional inequalities. In the Gaussian context this investigation sheds light to the…
Consider the problem of estimating the Shannon entropy of a distribution over $k$ elements from $n$ independent samples. We show that the minimax mean-square error is within universal multiplicative constant factors of $$\Big(\frac{k }{n…
Is there a natural way to order data in dimension greater than one? The approach based on the notion of data depth, often associated with John Tukey, is among the most popular. Tukey's depth has found applications in robust statistics,…