Related papers: On the Kozachenko-Leonenko entropy estimator
A problem of improving the accuracy of nonparametric entropy estimation for a stationary ergodic process is considered. New weak metrics are introduced and relations between metrics, measures, and entropy are discussed. Based on weak…
We introduce unbiased estimators for the Shannon entropy and the class number, in the situation that we are able to take sequences of independent samples of arbitrary length.
The Vapnik-Chervonenkis dimension of a set K in R^n is the maximal dimension of the coordinate cube of a given size, which can be found in coordinate projections of K. We show that the VC dimension of a convex body governs its entropy. This…
We compute Araki's relative entropy associated to a bounded interval $I=(a,b)$ between a thermal state and a coherent excitation of itself in the bosonic U(1)-current model, namely the (derivative of the) chiral boson. For this purpose we…
We analyze the Kozachenko--Leonenko (KL) nearest neighbor estimator for the differential entropy. We obtain the first uniform upper bound on its performance over H\"older balls on a torus without assuming any conditions on how close the…
A class of estimators of the R\'{e}nyi and Tsallis entropies of an unknown distribution $f$ in $\mathbb{R}^m$ is presented. These estimators are based on the $k$th nearest-neighbor distances computed from a sample of $N$ i.i.d. vectors with…
In this article we prove three fundamental types of limit theorems for the $q$-norm of random vectors chosen at random in an $\ell_p^n$-ball in high dimensions. We obtain a central limit theorem, a moderate deviations as well as a large…
We show how a central limit theorem for Poisson model random polygons implies a central limit theorem for uniform model random polygons. To prove this implication, it suffices to show that in the two models, the variables in question have…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
We study the problem of nonparametric estimation under $\bL_p$-loss, $p\in [1,\infty)$, in the framework of the convolution structure density model on $\bR^d$. This observation scheme is a generalization of two classical statistical models,…
This paper illustrates the power of the entropy method in addressing problems from large deviation theory. We provide and review entropy proofs for most fundamental results in large deviation theory, including Cramer's theorem, the…
The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…
We investigate the asymptotic behavior of the $L_p$-distance between a monotone function on a compact interval and a smooth estimator of this function. Our main result is a central limit theorem for the $L_p$-error of smooth isotonic…
General extensions of an inequality due to Rogozin, concerning the essential supremum of a convolution of probability density functions on the real line, are obtained. While a weak version of the inequality is proved in the very general…
For an expansive homeomorphism, we investigate the relationship among dimension, entropy, and Lyapunov exponents. Motivated by Young's formula for surface diffeomorphisms, which links dimension and measure-theoretic entropy with hyperbolic…
Let $\mathbb{B}_p^N$ be the $N$-dimensional unit ball corresponding to the $\ell_p$-norm. For each $N\in\mathbb N$ we sample a uniform random subspace $E_N$ of fixed dimension $m\in\mathbb{N}$ and consider the volume of $\mathbb{B}_p^N$…
From the analyticity properties of the equation governing infinitesimal perturbations, it is shown that all stability properties of spatially extended 1D systems can be derived from a single function that we call entropy potential since it…
This paper proposes two estimators of the joint entropy of the Type-II censored data. Consistency of both estimators is proved. Simulation results show that the second one shows less bias and root of mean square error (RMSE) than leading…
In this paper we study the notion of estimation entropy recently established by Liberzon and Mitra. This quantity measures the smallest rate of information about the state of a dynamical system above which an exponential state estimation…
We show that for every large enough integer $N$, there exists an $N$-point subset of $L_1$ such that for every $D>1$, embedding it into $\ell_1^d$ with distortion $D$ requires dimension $d$ at least $N^{\Omega(1/D^2)}$, and that for every…