相关论文: Shannon Entropy: Axiomatic Characterization and Ap…

Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…

Extropy was introduced as a dual complement of the Shannon entropy. In this investigation, we consider failure extropy and its dynamic version. Various basic properties of these measures are presented. It is shown that the dynamic failure…

Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…

Product probability property, known in the literature as statistical independence, is examined first. Then generalized entropies are introduced, all of which give generalizations to Shannon entropy. It is shown that the nature of the…

This paper is prepared as a contribution to the proceedings after the 12th ICSSUR/Feynfest Conference held in Foz do Iguacu (Brazil) from 2 to 6 May 2011. In the first part I briefy report the topic of entropic uncertainty relations for…

The Tsallis entropy given for a positive parameter $\alpha$ can be considered as a modification of the classical Shannon entropy. For the latter, corresponding to $\alpha=1$, there exist many axiomatic characterizations. One of them based…

We introduce a method for quantifying the inherent unpredictability of a continuous-valued time series via an extension of the differential Shannon entropy rate. Our extension, the specific entropy rate, quantifies the amount of predictive…

Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…

Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…

It is shown that the standard expression for the information entropy, originally due to Shannon, is only valid for a particular set of states. For the general case of statistical mechanics, one needs to include an additional term in the…

In this paper we remark that Shannon entropy can be expressed as a function of the self-information (i.e. the logarithm) and the inverse of the Lambert $W$ function. It means that we consider that Shannon entropy has the trace form: $-k…

We introduce unbiased estimators for the Shannon entropy and the class number, in the situation that we are able to take sequences of independent samples of arbitrary length.

There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the…

Entropic measures of complexity are able to quantify the information encoded in complex network structures. Several entropic measures have been proposed in this respect. Here we study the relation between the Shannon entropy and the Von…

We introduce a simple geometrical construction similar to covariant holographic entanglement entropy but with the addition of a new term proportional to boundary region volume. This new procedure has properties strongly resembling classical…

This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…

Thermodynamic uncertainty relations reveal a fundamental trade-off between the precision of a trajectory observable and entropy production, where the uncertainty of the observable is quantified by its variance. In information theory,…

The statistical analysis of data stemming from dynamical systems, including, but not limited to, time series, routinely relies on the estimation of information theoretical quantities, most notably Shannon entropy. To this purpose, possibly…

The estimation of entropy rates for stationary discrete-valued stochastic processes is a well studied problem in information theory. However, estimating the entropy rate for stationary continuous-valued stochastic processes has not received…

This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…