Related papers: On Generalized Sch\"urmann Entropy Estimators
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
Shannon's entropy and other entropy-based concepts are derived from the new, more general concept of relative divergence of one "grading' function on a linearly ordered set from another such function. The definition of relative divergence…
We present two new estimators for estimating the entropy of absolutely continuous random variables. Some properties of them are considered, specifically consistency of the first is proved. The introduced estimators are compared with the…
The statistical analysis of data stemming from dynamical systems, including, but not limited to, time series, routinely relies on the estimation of information theoretical quantities, most notably Shannon entropy. To this purpose, possibly…
Shannon entropy for discrete distributions is a fundamental and widely used concept, but its continuous analogue, known as differential entropy, lacks essential properties such as positivity and compatibility with the discrete case. In this…
In this paper, we develop the notion of entropy for uniform hypergraphs via tensor theory. We employ the probability distribution of the generalized singular values, calculated from the higher-order singular value decomposition of the…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
We conclude a sequence of work by giving near-optimal sketching and streaming algorithms for estimating Shannon entropy in the most general streaming model, with arbitrary insertions and deletions. This improves on prior results that obtain…
We consider the problem of mean estimation assuming only finite variance. We study a new class of mean estimators constructed by integrating over random noise applied to a soft-truncated empirical mean estimator. For appropriate choices of…
Alternative definitions are given of basic concepts of generalized thermostatistics. In particular, generalizations of Shannon's entropy, of the Boltzmann-Gibbs distribution, and of relative entropy are considered. Particular choices made…
This paper is a follow-up to Maruyama and Strawderman (2006, Journal of Statistical Planning and Inference), which identified a new class of generalized Bayes estimators with a particularly simple form for estimating a normal variance under…
The entropy of probability distribution defined by Shannon has several extensions. R\'enyi entropy is one of the general extensions of Shannon entropy and is widely used in engineering, physics, and so on. On the other hand, the quantum…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
We consider estimating the Shannon entropy of a discrete distribution $P$ from $n$ i.i.d. samples. Recently, Jiao, Venkat, Han, and Weissman, and Wu and Yang constructed approximation theoretic estimators that achieve the minimax $L_2$…
We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…
In this paper, we provide the R\'enyi entropy and complexity measure for a novel, flexible class of skew-gaussian distributions and their related families, as a characteristic form of the skew-gaussian Shannon entropy. We give closed…
Baez, Fritz, and Leinster derived a method for characterizing Shannon entropy in classical systems. In this method, they considered a functor from a certain category to the monoid of non-negative real numbers with addition as a map from…
We explore a possible application of the additive generalization of the Boltzmann-Gibbs-Shannon entropy proposed in [A.N. Gorban, I.V. Karlin, Phys. Rev. E, 67:016104 (2003)] to fully developed turbulence. The predicted probability…
We explore the relation between entanglement entropy of quantum many body systems and the distribution of corresponding, properly selected, observables. Such a relation is necessary to actually measure the entanglement entropy. We show that…