Related papers: Entropy modulo a prime
In this paper, we identify a class of absolutely continuous probability distributions, and show that the differential entropy is uniformly convergent over this space under the metric of total variation distance. One of the advantages of…
We introduce a binary relation on the finite discrete probability distributions which generalizes notions of majorization that have been studied in quantum information theory. Motivated by questions in thermodynamics, our relation describes…
We associate to the p-th R\'enyi entropy a definition of entropy power, which is the natural extension of Shannon's entropy power and exhibits a nice behaviour along solutions to the p-nonlinear heat equation in $R^n$. We show that the…
Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric…
There is a probability charge on the power set of the integers that gives probability $1/p$ to every residue class modulo a prime $p$. There exists such a charge that gives probability $w$ to the set of prime numbers iff $w \in [0,1/2]$.…
The aim of this note is to introduce a notion of dynamical entropy, which we call infinite-product entropy, for probability measures on (countable) infinite cartesian product of any measurable space with itself. The idea behind the…
Entropy has been a common index to quantify the complexity of time series in a variety of fields. Here, we introduce increment entropy to measure the complexity of time series in which each increment is mapped into a word of two letters,…
For \Gamma a countable amenable group consider those actions of \Gamma as measure-preserving transformations of a standard probability space, written as {T_\gamma}_{\gamma \in \Gamma} acting on (X,{\cal F}, \mu). We say…
Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies…
Comparison-based algorithms are algorithms for which the execution of each operation is solely based on the outcome of a series of comparisons between elements. Comparison-based computations can be naturally represented via the following…
In a previous paper, we proposed a probability interpretation for higher genus amplitudes of BMN (Berenstein-Maldacena-Nastase) strings in a pp-wave background with infinite negative curvature. This provides a natural definition of the…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
It is well-known that the Shannon entropies of some parameterized probability distributions are concave functions with respect to the parameter. In this paper we consider a family of such distributions (including the binomial, Poisson, and…
The entropy of probability distribution defined by Shannon has several extensions. R\'enyi entropy is one of the general extensions of Shannon entropy and is widely used in engineering, physics, and so on. On the other hand, the quantum…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
A new notion of partition-determined functions is introduced, and several basic inequalities are developed for the entropy of such functions of independent random variables, as well as for cardinalities of compound sets obtained using these…
In this paper we study certain properties of R\'{e}nyi entropy functionals $H_\alpha(\mathcal{P})$ on the space of probability distributions over $\mathbb{Z}_+$. Primarily, continuity and convergence issues are addressed. Some properties…
In classical thermodynamics the entropy is an extensive quantity, i.e.\ the sum of the entropies of two subsystems in equilibrium with each other is equal to the entropy of the full system consisting of the two subsystems. The extensitivity…
A definition of the thermodynamic entropy based on the time-dependent probability distribution of the macroscopic variables is developed. When a constraint in a composite system is released, the probability distribution for the new…