Related papers: The maximum entropy of a metric space
We develop a framework to extend resource measures from one domain to a larger one. We find that all extensions of resource measures are bounded between two quantities that we call the minimal and maximal extensions. We discuss various…
We develop information-theoretic measures of spatial structure and pattern in more than one dimension. As is well known, the entropy density of a two-dimensional configuration can be efficiently and accurately estimated via a converging…
The consistency of the species abundance distribution across diverse communities has attracted widespread attention. In this paper, I argue that the consistency of pattern arises because diverse ecological mechanisms share a common symmetry…
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
Observational entropy -- a quantity that unifies Boltzmann's entropy, Gibbs' entropy, von Neumann's macroscopic entropy, and the diagonal entropy -- has recently been argued to play a key role in a modern formulation of statistical…
We show that a simple geometric result suffices to derive the form of the optimal solution in a large class of finite and infinite-dimensional maximum entropy problems concerning probability distributions, spectral densities and covariance…
The concept of Shannon entropy of random variables was generalized to measurable functions in general, and to simple functions with finite values in particular. It is shown that the information measure of a function is related to the time…
Two new information-theoretic methods are introduced for establishing Poisson approximation inequalities. First, using only elementary information-theoretic techniques it is shown that, when $S_n=\sum_{i=1}^nX_i$ is the sum of the (possibly…
Our capacity to process information depends on the computational power at our disposal. Information theory captures our ability to distinguish states or communicate messages when it is unconstrained with unrivaled beauty and elegance. For…
The Maximum Entropy Principle (MEP) is a method that can be used to infer the value of an unknown quantity in a set of probability functions. In this work we review two applications of MEP: one giving a precise inference of the Higgs boson…
Suppose that you have $n$ colours and $m$ mutually independent dice, each of which has $r$ sides. Each dice lands on any of its sides with equal probability. You may colour the sides of each die in any way you wish, but there is one…
We shall prove that the celebrated R\'enyi entropy is the first example of a new family of infinitely many multi-parametric entropies. We shall call them the $Z$-entropies. Each of them, under suitable hypotheses, generalizes the celebrated…
The need to estimate smooth probability distributions (a.k.a. probability densities) from finite sampled data is ubiquitous in science. Many approaches to this problem have been described, but none is yet regarded as providing a definitive…
G. Edelman, O. Sporns, and G. Tononi have introduced the neural complexity of a family of random variables, defining it as a specific average of mutual information over subfamilies. We show that their choice of weights satisfies two natural…
The metric complexity (sometimes called Leinster--Cobbold maximum diversity) of a compact metric space is a recently introduced isometry-invariant of compact metric spaces which generalizes the notion of cardinality, and can be thought of…
We consider some classes of piecewise expanding maps in finite dimensional spaces having invariant probability measures which are absolutely continuous with respect to Lebesgue measure. We derive an entropy formula for such measures and,…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Maximum entropy distributions with discrete support in $m$ dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions…
We study maximum-entropy inference for finite-dimensional quantum states under linear moment constraints. Given expectation values of finitely many observables, the feasible set of states is convex but typically non-unique. The…
The paper makes the observation that all orders of information entropy are equal in signals composed of repeating units of distinct symbols where the units can be classified as a member of a symmetry group. This leads to an improved metric…