Related papers: General entropy-like uncertainty relations in fini…
We examine the {combinatorial} or {probabilistic} definition ("Boltzmann's principle") of the entropy or cross-entropy function $H \propto \ln \mathbb{W}$ or $D \propto - \ln \mathbb{P}$, where $\mathbb{W}$ is the statistical weight and…
In recent years, learning for neural networks can be viewed as optimization in the space of probability measures. To obtain the exponential convergence to the optimizer, the regularizing term based on Shannon entropy plays an important…
A number of recent works have sought to generalize the Kolmogorov-Sinai entropy of probability-preserving transformations to the setting of Markov operators acting on the integrable functions on a probability space $(X,\mu)$. These have…
We show that a proper expression of the uncertainty relation for a pair of canonically-conjugate continuous variables relies on entropy power, a standard notion in Shannon information theory for real-valued signals. The resulting…
We study coherent states associated to a generalized uncertainty principle (GUP). We separately analyze the cases of positive and negative deformation parameter $\beta$, showing that the ensuing probability distribution is a Tsallis…
We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, R\'enyi, generalized R\'enyi, Tsallis, Sharma-Mittal, and modified Shannon entropy,…
We define a general notion of entropy in elementary, algebraic terms. Based on that, weak forms of a scalar product and a distance measure are derived. We give basic properties of these quantities, generalize the Cauchy-Schwarz inequality,…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
Information entropies give a genuine way to characterize quantitatively an incompatibility in quantum measurements. Together with the Shannon entropy, few families of parametrized entropies have found use in various questions. It is also…
The coupled entropy is proven to correct a flaw in the derivation of the Tsallis entropy and thereby solidify the theoretical foundations for analyzing the uncertainty of complex systems. The Tsallis entropy originated from considering…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…
We prove several fundamental statistical bounds for entropic OT with the squared Euclidean cost between subgaussian probability measures in arbitrary dimension. First, through a new sample complexity result we establish the rate of…
We introduce a notion of a point-wise entropy of measures (i.e local entropy) called neutralized local entropy, and compare it with the Brin-Katok local entropy. We show that the neutralized local entropy coincides with Brin-Katok local…
We define the entropic bounds, i.e minimal uncertainty for pairs of unitary testers in distinguishing between unitary transformations not unlike the well known entropic bounds for observables. We show that in the case of specific sets of…
Numerous entropy-type characteristics (functionals) generalizing R\'enyi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and…
We derive a measurement-independent asymptotic continuity bound on the observational entropy for general POVM measurements, making essential use of its property of bounded concavity. The same insight is used to obtain continuity bounds for…
We present simple and computationally efficient nonparametric estimators of R\'enyi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over $\R^d$. The estimators are…
We find a sharp combinatorial bound for the metric entropy of sets in R^n and general classes of functions. This solves two basic combinatorial conjectures on the empirical processes. 1. A class of functions satisfies the uniform Central…
In a pair of recent articles [PRL 105 (2010) 041302 - arXiv:1005.1132; JHEP 1103 (2011) 056 - arXiv:1012.2867] two of the current authors have developed an entropy bound for equilibrium uncollapsed matter using only classical general…