Related papers: An approximate formula for the entropy of the nega…
We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1…
The negative binomial distribution (NBD) has been theorized to express a scale-invariant property of many-body systems and has been consistently shown to outperform other statistical models in both describing the multiplicity of…
In this paper, we propose a closed form approximation to the mean and variance of a new generalization of negative binomial (NGNB) distribution arising from the Extended COM-Poisson (ECOMP) distribution developed by Chakraborty and Imoto…
In this article, a generalized version of Negative binomial-beta exponential distribution with five parameters have been introduced. Some interesting submodels have been derived from it. A comprehensive mathematical treatment of proposed…
We design, implement and test a simple algorithm which computes the approximate entropy of a finite binary string of arbitrary length. The algorithm uses a weighted average of the Shannon Entropies of the string and all but the last binary…
This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…
Shannon entropy for discrete distributions is a fundamental and widely used concept, but its continuous analogue, known as differential entropy, lacks essential properties such as positivity and compatibility with the discrete case. In this…
We prove a Bernstein-type bound for the difference between the average of negative log-likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the…
The Shannon entropy, and related quantities such as mutual information, can be used to quantify uncertainty and relevance. However, in practice, it can be difficult to compute these quantities for arbitrary probability distributions,…
Recent theoretical results renewed the interest in charged particle multiplicity distributions. The Shannon entropy of such distributions is conjectured to be related to the entanglement or von Neumann entropy of partonic quantum system. In…
We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction…
The negative binomial distribution has been widely used as a more flexible model than the Poisson distribution for count data. However, when the true data-generating process is Poisson, it is often challenging to distinguish it from a…
It is well-known that the Shannon entropies of some parameterized probability distributions are concave functions with respect to the parameter. In this paper we consider a family of such distributions (including the binomial, Poisson, and…
The negative binomial distribution is self similar: If the spectrum over the whole rapidity range gives rise to a negative binomial, in absence of correlation and if the source is unique, also a partial range in rapidity gives rise to the…
Entropy is a fundamental property of both classical and quantum systems, spanning myriad theoretical and practical applications in physics and computer science. We study the problem of obtaining estimates to within a multiplicative factor…
We explore the relation between entanglement entropy of quantum many body systems and the distribution of corresponding, properly selected, observables. Such a relation is necessary to actually measure the entanglement entropy. We show that…
We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…
Reliable data-driven estimation of Shannon entropy from small data sets, where the number of examples is potentially smaller than the number of possible outcomes, is a critical matter in several applications. In this paper, we introduce a…
In this article, a new natural discrete analog of the one parameter polynomial exponential(OPPE) distribution as a mixture of a number of negative binomial distributions has been proposed and is called as a natural discrete one parameter…
We present a near-optimal quantum algorithm, up to logarithmic factors, for estimating the Shannon entropy in the quantum probability oracle model. Our approach combines the singular value separation algorithm with quantum amplitude…