Related papers: Average Entropy Functions
The concept of entropy, firstly introduced in information theory, rapidly became popular in many applied sciences via Shannon's formula to measure the degree of heterogeneity among observations. A rather recent research field aims at…
We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…
Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a…
Entropy is being used in physics, mathematics, informatics and in related areas to describe equilibration, dissipation, maximal probability states and optimal compression of information. The Gini index on the other hand is an established…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…
The result of performing integrations over connection type variables in the path integral for the discrete field theory may be poorly defined in the case of non-compact gauge group with the Haar measure exponentially growing in some…
We introduce the (private) entropy of a directed graph (in a new network coding sense) as well as a number of related concepts. We show that the entropy of a directed graph is identical to its guessing number and can be bounded from below…
Recently, the forward-backward and Douglas-Rachford envelope functions were proposed in the literature. The stationary points of these envelope functions have a close relationship with the solutions of the possibly nonsmooth optimization…
We prove that there is no algorithm to decide whether a given integer vector is in the closure of the entropic cone $\overline{\Gamma_{n}^{*}}$. Equivalently, there is no decision procedure to determine whether a given integer-valued…
New families of Fisher information and entropy power inequalities for sums of independent random variables are presented. These inequalities relate the information in the sum of $n$ independent random variables to the information contained…
We normalize the combinatorial Laplacian of a graph by the degree sum, look at its eigenvalues as a probability distribution and then study its Shannon entropy. Equivalently, we represent a graph with a quantum mechanical state and study…
We consider a class of simple one parameter families of interval maps, and we study how metric (resp. topological) entropy changes as the parameter varies. We show that in many cases the entropy displays a semi-regular behaviour, i.e. it is…
We present a new class of estimators of Shannon entropy for severely undersampled discrete distributions. It is based on a generalization of an estimator proposed by T. Schuermann, which itself is a generalization of an estimator proposed…
We examine the minimization of information entropy for measures on the phase space of bounded domains, subject to constraints that are averages of grand canonical distributions. We describe the set of all such constraints and show that it…
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
Trigonometric and trigonometric-algebraic entropies are introduced. Regularity increases the entropy and the maximal entropy is shown to result when a regular $n$-gon is inscribed in a circle. A regular $n$-gon circumscribing a circle gives…
A central issue of the science of complex systems is the quantitative characterization of complexity. In the present work we address this issue by resorting to information geometry. Actually we propose a constructive way to associate to a -…
The entropic region is formed by the collection of the Shannon entropies of all subvectors of finitely many jointly distributed discrete random variables. For four or more variables, the structure of the entropic region is mostly unknown.…
We study a quantity called discrete layered entropy, which approximates the Shannon entropy within a logarithmic gap. Compared to the Shannon entropy, the discrete layered entropy is piecewise linear, approximates the expected length of the…
The entropy of a pair of random variables is commonly depicted using a Venn diagram. This representation is potentially misleading, however, since the multivariate mutual information can be negative. This paper presents new measures of…