English
Related papers

Related papers: Dimension-Free Empirical Entropy Estimation

200 papers

We extend previously proposed measures of complexity, emergence, and self-organization to continuous distributions using differential entropy. This allows us to calculate the complexity of phenomena for which distributions are known. We…

Adaptation and Self-Organizing Systems · Physics 2016-04-01 Guillermo Santamaría-Bonfil , Nelson Fernández , Carlos Gershenson

We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…

Quantum Physics · Physics 2014-02-19 F. Dupuis , L. Kraemer , P. Faist , J. M. Renes , R. Renner

Entropy estimation, due in part to its connection with mutual information, has seen considerable use in the study of time series data including causality detection and information flow. In many cases, the entropy is estimated using…

Statistics Theory · Mathematics 2019-08-06 Alexander L Young , David B Dunson

Entropy is a fundamental concept from Thermodynamics and it can be used to study models on context of Creation Cold Dark Matter (CCDM). From conditions on the first ($\dot{S}\geq0$)\footnote{Throughout the present work we will use dots to…

General Relativity and Quantum Cosmology · Physics 2020-12-02 R. Valentim , J. F. Jesus

We prove several fundamental statistical bounds for entropic OT with the squared Euclidean cost between subgaussian probability measures in arbitrary dimension. First, through a new sample complexity result we establish the rate of…

Statistics Theory · Mathematics 2019-05-31 Gonzalo Mena , Jonathan Weed

We establish weak limits for the empirical entropy regularized optimal transport cost, the expectation of the empirical plan and the conditional expectation. Our results require only uniform boundedness of the cost function and no…

Statistics Theory · Mathematics 2023-05-18 Alberto González-Sanz , Shayan Hundrieser

We leverage the Gibbs inequality and its natural generalization to R\'enyi entropies to derive closed-form parametric expressions of the optimal lower bounds of $\rho$th-order guessing entropy (guessing moment) of a secret taking values on…

Information Theory · Computer Science 2024-01-31 Julien Béguinot , Olivier Rioul

We consider the problem of estimating the population probability distribution given a finite set of multivariate samples, using the maximum entropy approach. In strict keeping with Jaynes' original definition, our precise formulation of the…

Data Analysis, Statistics and Probability · Physics 2007-07-13 Sabbir Rahman , Mahbub Majumdar

We study a curve of Gibbsian families of complex 3x3-matrices and point out new features, absent in commutative finite-dimensional algebras: a discontinuous maximum-entropy inference, a discontinuous entropy distance and non-exposed faces…

Mathematical Physics · Physics 2016-05-17 Andreas Knauf , Stephan Weis

A surrogate data analysis is presented, which is based on the fluctuations of the ``entropy'' $S$ defined in the natural time-domain [Phys. Rev. E {\bf 68}, 031106, 2003]. This entropy is not a static one as, for example, the Shannon…

Data Analysis, Statistics and Probability · Physics 2007-05-23 P. A. Varotsos , N. V. Sarlis , E. S. Skordas , M. S. Lazaridou

We present a technique for entropy optimization to calculate a distribution from its moments. The technique is based upon maximizing a discretized form of the Shannon entropy functional by mapping the problem onto a dual space where an…

Disordered Systems and Neural Networks · Physics 2009-11-10 K. Bandyopadhyay , A. K. Bhattacharya , Parthapratim Biswas , D. A. Drabold

We present differentially private algorithms for high-dimensional mean estimation. Previous private estimators on distributions over $\mathbb{R}^d$ suffer from a curse of dimensionality, as they require $\Omega(d^{1/2})$ samples to achieve…

Machine Learning · Computer Science 2024-11-04 Yuval Dagan , Michael I. Jordan , Xuelin Yang , Lydia Zakynthinou , Nikita Zhivotovskiy

During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…

Statistical Mechanics · Physics 2020-04-22 Mengjie Zu , Arunkumar Bupathy , Daan Frenkel , Srikanth Sastry

We consider the task of estimating a conditional density using i.i.d. samples from a joint distribution, which is a fundamental problem with applications in both classification and uncertainty quantification for regression. For joint…

Statistics Theory · Mathematics 2023-06-16 Blair Bilodeau , Dylan J. Foster , Daniel M. Roy

The uncertainty principle brings out intrinsic quantum bounds on the precision of measuring non-commuting observables. Statistical outcomes in the measurement of incompatible observables reveal a trade-off on the sum of corresponding…

Quantum Physics · Physics 2013-11-11 H. S. Karthik , A. R. Usha Devi , J. Prabhu Tej , A. K. Rajagopal

Let X_1, ..., X_n be a sequence of n classical random variables and consider a sample of r positions selected at random. Then, except with (exponentially in r) small probability, the min-entropy of the sample is not smaller than, roughly, a…

Quantum Physics · Physics 2012-06-04 Robert Koenig , Renato Renner

By proving that certain free stochastic differential equations have stationary solutions, we give a lower estimate on the microstates free entropy dimension of certain $n$-tuples $X_{1},...,X_{n}$: we show that Abstract. By proving that…

Operator Algebras · Mathematics 2008-07-03 D. Shlyakhtenko

We show that a certain type of quasi finite, conservative, ergodic, measure preserving transformation always has a maximal zero entropy factor, generated by predictable sets. We also construct a conservative, ergodic, measure preserving…

Dynamical Systems · Mathematics 2010-06-01 Jon Aaronson , Kyewon Koh Park

This paper develops a theory of distribution- and time-uniform asymptotics, culminating in the first large-sample anytime-valid inference procedures that are shown to be uniformly valid in a rich class of distributions. Historically,…

Statistics Theory · Mathematics 2026-01-16 Ian Waudby-Smith , Edward H. Kennedy , Aaditya Ramdas

This paper studies the complexity of estimating Renyi divergences of discrete distributions: $p$ observed from samples and the baseline distribution $q$ known \emph{a priori}. Extending the results of Acharya et al. (SODA'15) on estimating…

Information Theory · Computer Science 2017-02-09 Maciej Skorski