Related papers: An entropy functional bounded from above by one
The organization of high-dimensional probability spaces is a fundamental problem at the intersection of statistical physics and information theory. Here, we analyze the distributions populating level surfaces of the probability simplex…
In this article we introduce an entropy-based, scale-dependent centrality that is evaluated as the Shannon entropy of the distribution at time t of a continuous-time random walk. It ranks nodes as a function of the time t, which acts as a…
The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
We show how the Shannon entropy function can be used as a basis to set up complexity measures weighting the economic efficiency of countries and the specialization of products beyond bare diversification. This entropy function guarantees…
Estimation of Shannon and R\'enyi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on…
We show how generalized Gibbs-Shannon entropies can provide new insights on the statistical properties of texts. The universal distribution of word frequencies (Zipf's law) implies that the generalized entropies, computed at the word level,…
Baez, Fritz, and Leinster derived a method for characterizing Shannon entropy in classical systems. In this method, they considered a functor from a certain category to the monoid of non-negative real numbers with addition as a map from…
Thermodynamic uncertainty relations reveal a fundamental trade-off between the precision of a trajectory observable and entropy production, where the uncertainty of the observable is quantified by its variance. In information theory,…
Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering to physics, and obtaining bounds on the entropy of probability distributions is of fundamental importance in…
Under the scenario of generalized measurements, it can be questioned how much of quantum uncertainty can be attributed to measuring device, independent of the uncertainty in the measured system. On the course to answer the question, we…
This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon's entropy function has a…
We consider the problem of approximating the empirical Shannon entropy of a high-frequency data stream under the relaxed strict-turnstile model, when space limitations make exact computation infeasible. An equivalent measure of entropy is…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
The new estimates of the conditional Shannon entropy are introduced in the framework of the model describing a discrete response variable depending on a vector of d factors having a density w.r.t. the Lebesgue measure in R^d. Namely, the…
In this paper we will analyze discrete probability distributions in which probabilities of particular outcomes of some experiment (microstates) can be represented by the ratio of natural numbers (in other words, probabilities are…
Uncertainty relations are central to quantum physics. While they were originally formulated in terms of variances, they have later been successfully expressed with entropies following the advent of Shannon information theory. Here, we…
A surrogate data analysis is presented, which is based on the fluctuations of the ``entropy'' $S$ defined in the natural time-domain [Phys. Rev. E {\bf 68}, 031106, 2003]. This entropy is not a static one as, for example, the Shannon…
Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…
Any continuous curve in a higher dimensional space can be considered a trajectory that can be parameterized by a single variable, usually taken as time. It is well known that a continuous curve can have a fractional dimensionality, which…