Related papers: An entropy functional bounded from above by one
A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of…
In the present paper, we consider the plug-in estimator of Shannon's entropy defined on a finite alphabet which is assumed to dynamically vary as the sample size increases. The asymptotic behaviors for the plug-in estimator, such as,…
Building on the recent work of Johnson (2007) and Yu (2008), we prove that entropy is a concave function with respect to the thinning operation T_a. That is, if X and Y are independent random variables on Z_+ with ultra-log-concave…
We construct the generalized entropy optimized by a given arbitrary statistical distribution with a finite linear expectation value of a random quantity of interest. This offers, via the maximum entropy principle, a unified basis for a…
We prove a variety of new and refined uniform continuity bounds for entropies of both classical random variables on an infinite state space and of quantum states of infinite-dimensional systems. We obtain the first tight continuity estimate…
We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1…
In arXiv:1801.01238 a variation of Bowen's topological entropy that can be applied to the study of discontinuous semiflows on compact metric spaces was introduced. The main novetly is the use of certain family of pseudosemimetrics…
We introduce the (private) entropy of a directed graph (in a new network coding sense) as well as a number of related concepts. We show that the entropy of a directed graph is identical to its guessing number and can be bounded from below…
This study introduces the syntropy function ($S_N$) and expectancy function ($E_N$), derived from the novel function $\phi$, to provide a refined perspective on complexity, extending beyond conventional entropy analysis. $S_N$ is designed…
The Shannon entropy is a fundamental measure for quantifying diversity and model complexity in fields such as information theory, ecology, and genetics. However, many existing studies assume that the number of species is known, an…
Topological entropy is a widely studied indicator of chaos in topological dynamics. Here we give a generalized definition of topological entropy which may be applied to set-valued functions. We demonstrate that some of the well-known…
We revisit the well-studied problem of estimating the Shannon entropy of a probability distribution, now given access to a probability-revealing conditional sampling oracle. In this model, the oracle takes as input the representation of a…
Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are…
The R\'enyi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic R\'enyi…
Entropy has been a common index to quantify the complexity of time series in a variety of fields. Here, we introduce increment entropy to measure the complexity of time series in which each increment is mapped into a word of two letters,…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
In this article we recover the distribution function (and possible density) of an arbitrary random variable that is subject to an additive measurement error. This problem is also known as deconvolution and has a long tradition in…
The behavior of the Kozachenko - Leonenko estimates for the (differential) Shannon entropy is studied when the number of i.i.d. vector-valued observations tends to infinity. The asymptotic unbiasedness and L^2-consistency of the estimates…
The increased uncertainty and complexity of nonlinear systems have motivated investigators to consider generalized approaches to defining an entropy function. New insights are achieved by defining the average uncertainty in the probability…
Gamma distributions, which contain the exponential as a special case, have a distinguished place in the representation of near-Poisson randomness for statistical processes; typically, they represent distributions of spacings between events…