Related papers: Entropy modulo a prime
We describe five types of results concerning information and concentration of discrete random variables, and relationships between them, motivated by their counterparts in the continuous case. The results we consider are information…
Superstatistics describes statistical systems that behave like superpositions of different inverse temperatures $\beta$, so that the probability distribution is $p(\epsilon_i) \propto \int_{0}^{\infty} f(\beta) e^{-\beta \epsilon_i}d\beta$,…
A definition of entropy via the Kolmogorov algorithmic complexity is discussed. As examples, we show how the meanfield theory for the Ising model, and the entropy of a perfect gas can be recovered. The connection with computations are…
Fix a modulus $q$. One would expect the number of primes in each invertible residue class mod $q$ to be multinomially distributed, i.e. for each $p \,\mathrm{mod}\, q$ to behave like an independent random variable uniform on…
Markov categories are a novel framework to describe and treat problems in probability and information theory. In this work we combine the categorical formalism with the traditional quantitative notions of entropy, mutual information, and…
The Random Permutation Set (RPS) is a new type of set proposed recently, which can be regarded as the generalization of evidence theory. To measure the uncertainty of RPS, the entropy of RPS and its corresponding maximum entropy have been…
The entropy region is a fundamental object in information theory. An outer bound for the entropy region is defined by a minimal set of Shannon-type inequalities called elemental inequalities also referred to as the Shannon region. This…
We prove a lower estimate on the increase in entropy when two copies of a conditional random variable $X | Y$, with $X$ supported on $\mathbb{Z}_q=\{0,1,\dots,q-1\}$ for prime $q$, are summed modulo $q$. Specifically, given two i.i.d copies…
The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…
In the present paper we prove a family of tight upper and lower bounds for the Shannon entropy and von Neumann entropy based on the p-norms. This allows us to have an entropy estimate, a criterion for the finiteness of it and a bound on the…
General probabilistic theories are designed to provide operationally the most general probabilistic models including both classical and quantum theories. In this letter, we introduce a systematic method to construct a series of entropies,…
We discuss how the method of maximum entropy, MaxEnt, can be extended beyond its original scope, as a rule to assign a probability distribution, to a full-fledged method for inductive inference. The main concept is the (relative) entropy…
We present a new class of estimators of Shannon entropy for severely undersampled discrete distributions. It is based on a generalization of an estimator proposed by T. Schuermann, which itself is a generalization of an estimator proposed…
Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…
The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…
We provide, under minimal continuity assumptions, a description of \textsl{additive partition entropies}. They are real functions $I$ on the set of finite partitions that are additive on stochastically independent partitions in a given…
Algebraically, entropy can be defined for abelian groups and their endomorphisms, and was latter extended to consider objects in a Flow category derived from abelian categories, such as $R\textit{-}Mod$ with $R$ a ring. Preradicals are…
Matroidal entropy functions are entropy functions in the form $\mathbf{h} = \log v \cdot \mathbf{r}_M$ , where $v \ge 2$ is an integer and $\mathbf{r}_M$ is the rank function of a matroid $M$. They can be applied into capacity…
The microscopic explanation of entropy has been challenged from both experimental and theoretical point of view. The expression of entropy is derived from the first law of thermodynamics indicating that entropy or the second law of…
We consider distributions of ordered random vectors with given one-dimensional marginal distributions. We give an elementary necessary and sufficient condition for the existence of such a distribution with finite entropy. In this case, we…