English
Related papers

Related papers: Statistical estimation of the Shannon entropy

200 papers

We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…

Information Theory · Computer Science 2010-10-21 Oliver Johnson , Yaming Yu

We consider the problem of approximating the empirical Shannon entropy of a high-frequency data stream under the relaxed strict-turnstile model, when space limitations make exact computation infeasible. An equivalent measure of entropy is…

Computation · Statistics 2013-04-18 Peter Clifford , Ioana Ada Cosma

Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical…

Statistical Mechanics · Physics 2017-09-13 S. E. Marzen , J. P. Crutchfield

Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…

Information Theory · Computer Science 2024-05-07 Mokshay Madiman , Prasad Tetali

In this paper, we estimate the Shannon entropy $S(f) = -\E[ \log (f(x))]$ of a one-sided linear process with probability density function $f(x)$. We employ the integral estimator $S_n(f)$, which utilizes the standard kernel density…

Statistics Theory · Mathematics 2020-09-10 Timothy Fortune , Hailin Sang

Statistical properties of coupled dynamic-stochastic systems are studied within a combination of the maximum information principle and the superstatistical approach. The conditions at which the Shannon entropy functional leads to a…

Statistical Mechanics · Physics 2009-11-11 E. V. Vakarin , J. P. Badiali

A Massey-like inequality is any useful lower bound on guessing entropy in terms of the computationally scalable Shannon entropy. The asymptotically optimal Massey-like inequality is determined and further refined for finite-support…

Information Theory · Computer Science 2021-03-30 Andrei Tănăsescu , Marios O. Choudary , Olivier Rioul , Pantelimon George Popescu

We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction…

Statistical Mechanics · Physics 2009-11-10 Thomas Schürmann

This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…

Statistics Theory · Mathematics 2019-06-04 Ziv Goldfeld , Kristjan Greenewald , Yury Polyanskiy

It is not obvious how to extend Shannon's original information entropy to higher dimensions, and many different approaches have been tried. We replace the English text symbol sequence originally used to illustrate the theory by a discrete,…

Information Theory · Computer Science 2016-09-06 Kieran G. Larkin

We suppose that a L\'evy process is observed at discrete time points. A rather general construction of minimum-distance estimators is shown to give consistent estimators of the L\'evy-Khinchine characteristics as the number of observations…

Statistics Theory · Mathematics 2008-05-29 Michael H. Neumann , Markus Reiss

According to a previous conjecture, spatial and temporal Lyapunov exponents of chaotic extended systems can be obtained from derivatives of a suitable function: the entropy potential. The validity and the consequences of this hypothesis are…

chao-dyn · Physics 2009-10-30 Stefano Lepri , Antonio Politi , Alessandro Torcini

We evaluate Shannon entropy for the position and momentum eigenstates of some conditionally exactly solvable potentials which are isospectral to harmonic oscillator and whose solutions are given in terms of exceptional orthogonal…

Mathematical Physics · Physics 2015-05-27 D. Dutta , P. Roy

The Lesche stability condition for the Shannon entropy [B. Lesche, J. Stat. Phys. 27, 419 (1982)], represents a fundamental test, for its experimental robustness, for systems obeying the Maxwell-Boltzmann statistical mechanics. Of course,…

Statistical Mechanics · Physics 2009-11-10 G. Kaniadakis , A. M. Scarfone

There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the…

Information Theory · Computer Science 2017-08-22 John C. Baez , Tobias Fritz , Tom Leinster

Max stable laws are limit laws of linearly normalized partial maxima of indepen- dent, identically distributed (iid) random variables (rvs). These are analogous to stable laws which are limit laws of normalized partial sums of iid rvs. In…

Statistics Theory · Mathematics 2016-11-25 Sreenivasan Ravi , Ali Saeb

The entropic region is formed by the collection of the Shannon entropies of all subvectors of finitely many jointly distributed discrete random variables. For four or more variables, the structure of the entropic region is mostly unknown.…

Information Theory · Computer Science 2026-03-04 E. P. Csirmaz , L. Csirmaz

We consider the concept of generalized measure-theoretic entropy, where instead of the Shannon entropy function we consider an arbitrary concave function defined on the unit interval, vanishing in the origin. Under mild assumptions on this…

Dynamical Systems · Mathematics 2014-08-21 Fryderyk Falniowski

In this paper we remark that Shannon entropy can be expressed as a function of the self-information (i.e. the logarithm) and the inverse of the Lambert $W$ function. It means that we consider that Shannon entropy has the trace form: $-k…

Statistical Mechanics · Physics 2019-07-05 Laurent Truffet

Let $ X_1, \ldots, X_n $ be independent random variables taking values in the alphabet $ \{0, 1, \ldots, r\} $, and $ S_n = \sum_{i = 1}^n X_i $. The Shepp--Olkin theorem states that, in the binary case ($ r = 1 $), the Shannon entropy of $…

Information Theory · Computer Science 2022-05-10 Mladen Kovačević