Related papers: A Tight Uniform Continuity Bound for Equivocation
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
This paper focuses on the extreme-value problem for Shannon entropy of the joint distribution with given marginals. It is proved that the minimum-entropy coupling must be of order-preserving, while the maximum-entropy coupling coincides…
We bound the rate of uniform convergence in compact sets for both entropic potentials and their gradients towards the Brenier potential and its gradient, respectively. Both results hold in the quadratic Euclidean setting for absolutely…
We derive a universal bound on the large-deviation functions of particle currents in coherent conductors. This bound depends only on the mean value of the relevant current and the total rate of entropy production required to maintain a…
We explore asymptotically optimal bounds for deviations of distributions of independent Bernoulli random variables from the Poisson limit in terms of the Shannon relative entropy and R\'enyi/Tsallis relative distances (including Pearson's…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
We introduce the notion of uniform exactness, or uniform amenability at infinity, for discrete groups and prove it for a wide class of groups containing free groups and their limit groups. This shows a novel strong convergence phenomenon…
The Shannon-Khinchin axioms for the ordinary information entropy are generalized in a natural way to the nonextensive systems based on the concept of nonextensive conditional entropy, and a complete proof of the uniqueness theorem for the…
Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…
The thermodynamic uncertainty relation offers a universal energetic constraint on the relative magnitude of current fluctuations in nonequilibrium steady states. However, it has only been derived for long observation times. Here, we prove a…
A sufficient condition for the uniform modulus of continuity of a random field $X = \{X(t), t \in \R^N\}$ is provided. The result is applicable to random fields with heavy-tailed distribution such as stable random fields.
The paper examines relationships between the conditional Shannon entropy and the expectation of $\ell_{\alpha}$-norm for joint probability distributions. More precisely, we investigate the tight bounds of the expectation of…
Entropic uncertainty relations for the position and momentum within the generalized uncertainty principle are examined. Studies of this principle are motivated by the existence of a minimal observable length. Then the position and momentum…
The uncertainty relation for continuous variables due to Byalinicki-Birula and Mycielski expresses the complementarity between two $n$-uples of canonically conjugate variables $(x_1,x_2,\cdots x_n)$ and $(p_1,p_2,\cdots p_n)$ in terms of…
In the present work we investigate phase correlations by recourse to the Shannon entropy. Using theoretical arguments we show that the entropy provides an accurate measure of phase correlations in any dynamical system, in particular when…
We show that Frenkel's integral representation of the quantum relative entropy provides a natural framework to derive continuity bounds for quantum information measures. Our main general result is a dimension-independent semi-continuity…
Thermodynamic uncertainty relations reveal a fundamental trade-off between the precision of a trajectory observable and entropy production, where the uncertainty of the observable is quantified by its variance. In information theory,…
Relations between Shannon entropy and Renyi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Renyi entropies of order two and three are known, we provide an lower and an upper bound…
Reliable data-driven estimation of Shannon entropy from small data sets, where the number of examples is potentially smaller than the number of possible outcomes, is a critical matter in several applications. In this paper, we introduce a…
Edgeworth-type expansions for convolutions of probability densities and powers of the characteristic functions with non-uniform error terms are established for i.i.d. random variables with finite (fractional) moments of order $s \geq 2$,…