English
Related papers

Related papers: Statistical estimation of the Shannon entropy

200 papers

We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1…

Data Analysis, Statistics and Probability · Physics 2011-11-09 P. Grassberger

We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…

Quantum Physics · Physics 2007-05-23 C. G. Chakrabarti , Indranil Chakrabarty

In this paper, we investigate the asymptotic stability of finite-dimensional stochastic integrable Hamiltonian systems via information entropy. Specifically, we establish the asymptotic vanishing of Shannon entropy difference (with…

Dynamical Systems · Mathematics 2025-10-28 Chen Wang , Yong Li

We calculate Shannon information entropy of trapped interacting bosons in both the position and momentum spaces, $S_r$ and $S_k$ respectively. The total entropy maintains the fuctional form $S=a + b \ln N$ for repulsive bosons. At the…

Quantum Gases · Physics 2013-01-25 Sudip Kumar Haldar , Barnali Chakrabarti

We consider stationary ergodic processes indexed by $\mathbb Z$ or $\mathbb Z^n$ whose finite dimensional marginals have laws which are absolutely continuous with respect to Lebesgue measure. We define an entropy theory for these continuous…

Dynamical Systems · Mathematics 2007-05-23 D. Hamdan , W. Parry , J. -P. Thouvenot

This paper derives bounds for two omnipresent information theoretic measures, the Shannon entropy and its complementary dual, the extropy. Based on a large size data set from a logconcave model, the said bounds are obtained for the entropy…

Statistics Theory · Mathematics 2025-07-21 Konstantinos Zografos

Reliable data-driven estimation of Shannon entropy from small data sets, where the number of examples is potentially smaller than the number of possible outcomes, is a critical matter in several applications. In this paper, we introduce a…

Machine Learning · Computer Science 2025-12-12 Gabriel F. A. Bastos , Jugurta Montalvão

This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…

Statistics Theory · Mathematics 2025-12-03 Octavio César Mesner

Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…

Information Theory · Computer Science 2022-07-26 John Çamkıran

Estimating the Shannon entropy of a discrete distribution from which we have only observed a small sample is challenging. Estimating other information-theoretic metrics, such as the Kullback-Leibler divergence between two sparsely sampled…

Data Analysis, Statistics and Probability · Physics 2023-02-24 Angelo Piga , Lluc Font-Pomarol , Marta Sales-Pardo , Roger Guimerà

Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…

Statistics Theory · Mathematics 2022-05-25 Jialin Zhang , Jingyi Shi

We introduce a novel entropy-related function, \textit{non-repeatability}, designed to capture dynamical behaviors in complex systems. Its normalized form, \textit{mutability}, has been previously applied in statistical physics as a…

Statistical Mechanics · Physics 2025-04-04 Eugenio E. Vogel , Francisco J. Peña , G. Saravia , P. Vargas

A method of estimating the joint probability mass function of a pair of discrete random variables is described. This estimator is used to construct the conditional Shannon-R\'eyni-Tsallis entropies estimates. From there almost sure rates of…

Statistics Theory · Mathematics 2020-02-18 Ba Amadou Diadie , Lo Gane Samb

The Shannon entropy is used as a basis for applying different lemmas and conjectures concerning the set of gaps between prime numbers G_p , thus estimating several measures of it. The same procedures are applied to artificially created…

General Mathematics · Mathematics 2016-06-28 Arturo Ortiz Tapia , Hans Henrik Støleum

We consider the uncertainty between two pairs of local projective measurements performed on a multipartite system. We show that the optimal bound in any linear uncertainty relation, formulated in terms of the Shannon entropy, is additive.…

Quantum Physics · Physics 2018-03-28 Rene Schwonnek

Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…

Statistical Mechanics · Physics 2013-04-04 Valentina Baccetti , Matt Visser

The entropic uncertainty measures of the multidimensional hydrogenic states quantify the multiple facets of the spatial delocalization of the electronic probability density of the system. The Shannon entropy is the most adequate uncertainty…

Quantum Physics · Physics 2019-11-19 Irene V. Toranzo , David Puertas-Centeno , Nahual Sobrino , Jesús S. Dehesa

Shannon entropy is often a quantity of interest to linguists studying the communicative capacity of human language. However, entropy must typically be estimated from observed data because researchers do not have access to the underlying…

Computation and Language · Computer Science 2022-04-06 Aryaman Arora , Clara Meister , Ryan Cotterell

We investigate the behavior of the Gibbs-Shannon entropy of the stationary nonequilibrium measure describing a one-dimensional lattice gas, of L sites, with symmetric exclusion dynamics and in contact with particle reservoirs at different…

Statistical Mechanics · Physics 2015-05-13 B. Derrida , J. L. Lebowitz , E. R. Speer

Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…

Quantum Physics · Physics 2022-09-07 Alexey E. Rastegin