English
Related papers

Related papers: Dimension-Free Empirical Entropy Estimation

200 papers

A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of…

Probability · Mathematics 2021-06-02 Lampros Gavalakis , Ioannis Kontoyiannis

Order statistics theory is applied in this paper to probabilistic robust control theory to compute the minimum sample size needed to come up with a reliable estimate of an uncertain quantity under continuity assumption of the related…

Optimization and Control · Mathematics 2008-05-13 Xinjia Chen , Kemin Zhou

The method of maximum entropy is quite a powerful tool to solve the generalized moment problem, which consists of determining the probability density of a random variable X from the knowledge of the expected values of a few functions of the…

Statistics Theory · Mathematics 2015-10-15 Henryk Gzyl

We derive a lower bound on the smallest output entropy that can be achieved via vector quantization of a $d$-dimensional source with given expected $r$th-power distortion. Specialized to the one-dimensional case, and in the limit of…

Information Theory · Computer Science 2017-03-27 Tobias Koch , Gonzalo Vazquez-Vilar

We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies…

Metric Geometry · Mathematics 2020-12-17 Tom Leinster , Emily Roff

Measuring entropy production of a system directly from the experimental data is highly desirable since it gives a quantifiable measure of the time-irreversibility for non-equilibrium systems and can be used as a cost function to optimize…

Statistical Mechanics · Physics 2024-07-16 Prashant Singh , Karel Proesmans

We study the problem of robustly estimating the mean or location parameter without moment assumptions. We show that for a large class of symmetric distributions, the same error as in the Gaussian setting can be achieved efficiently. The…

Data Structures and Algorithms · Computer Science 2023-11-09 Gleb Novikov , David Steurer , Stefan Tiegel

In this work we introduce a method for estimating entropy rate and entropy production rate from finite symbolic time series. From the point of view of statistics, estimating entropy from a finite series can be interpreted as a problem of…

Statistical Mechanics · Physics 2021-02-24 Raul Salgado-Garcia , Cesar Maldonado

This work contains two single-letter upper bounds on the entropy rate of a discrete-valued stationary stochastic process, which only depend on second-order statistics, and are primarily suitable for models which consist of relatively large…

Information Theory · Computer Science 2022-03-11 Ran Tamir

Recently proposed generative models for discrete data, such as Masked Diffusion Models (MDMs), exploit conditional independence approximations to reduce the computational cost of popular Auto-Regressive Models (ARMs), at the price of some…

Machine Learning · Statistics 2025-12-18 Hugo Lavenant , Giacomo Zanella

Sequential recommender systems have achieved steady gains in offline accuracy, yet it remains unclear how close current models are to the intrinsic accuracy limit imposed by the data. A reliable, model-agnostic estimate of this ceiling…

Information Retrieval · Computer Science 2026-04-15 En Xu , Jingtao Ding , Yong Li

Motivated by the entropy computations relevant to the evaluation of decrease in entropy in bit reset operations, the authors investigate the deficit in an entropic inequality involving two independent random variables, one continuous and…

Information Theory · Computer Science 2018-09-21 James Melbourne , Saurav Talukdar , Shreyas Bhaban , Murti V. Salapaka

The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…

Data Analysis, Statistics and Probability · Physics 2009-09-29 Ilya Nemenman , William Bialek , Rob de Ruyter van Steveninck

We produce a probabilistic space from logic, both classical and quantum, which is in addition partially ordered in such a way that entropy is monotone. In particular do we establish the following equation: Quantitative Probability = Logic +…

Quantum Physics · Physics 2009-09-29 Bob Coecke

Determining the strength of non-linear statistical dependencies between two variables is a crucial matter in many research fields. The established measure for quantifying such relations is the mutual information. However, estimating mutual…

Data Analysis, Statistics and Probability · Physics 2019-07-24 Damián G. Hernández , Inés Samengo

We perform Transition matrix Monte Carlo simulations to evaluate the entropy of rhombus tilings with fixed polygonal boundaries and 2D-fold rotational symmetry. We estimate the large-size limit of this entropy for D=4 to 10. We confirm…

Statistical Mechanics · Physics 2009-11-11 Nicolas Destainville

Heisenberg's uncertainty principle has recently led to general measurement uncertainty relations for quantum systems: incompatible observables can be measured jointly or in sequence only with some unavoidable approximation, which can be…

Quantum Physics · Physics 2017-06-27 Alberto Barchielli , Matteo Gregoratti , Alessandro Toigo

In this work, we consider a recently proposed entropy S (called varentropy) defined by a variational relationship dI=beta*(d<x>-<dx>) as a measure of uncertainty of random variable x. By definition, varentropy underlies a generalized…

Statistical Mechanics · Physics 2020-10-28 C. J. Ou , A. El Kaabouchi , L. Nivanen , F. Tsobnang , A. Le Méhauté , Qiuping A. Wang

In data analysis problems where we are not able to rely on distributional assumptions, what types of inference guarantees can still be obtained? Many popular methods, such as holdout methods, cross-validation methods, and conformal…

Statistics Theory · Mathematics 2022-05-31 Yonghoon Lee , Rina Foygel Barber

A fundamental problem in statistics is estimating the shape matrix of an Elliptical distribution. This generalizes the familiar problem of Gaussian covariance estimation, for which the sample covariance achieves optimal estimation error.…

Statistics Theory · Mathematics 2025-10-16 Lap Chi Lau , Akshay Ramachandran