Related papers: Entropy modulo a prime
We introduce a canonical notion of entropy for polynomials analogue to that of random variables in probability. We prove that entropy increases smoothly with respect to finite free addition. In particular we get the new inequality : $…
There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…
We propose a new way of defining entropy of a system, which gives a general form which may be nonextensive as Tsallis entropy, but is linearly dependent on component entropies, like Renyi entropy, which is extensive. This entropy has a…
For certain groups, parabolic subgroups appear as stabilizers of flags of sets or vector spaces. Quotients by these parabolic subgroups represent orbits of flags, and their cardinalities asymptotically reveal entropies (as rates of…
We quantify a social organization's potentiality, that is its ability to attain different configurations. The organization is represented as a network in which nodes correspond to individuals and (multi-)edges to their multiple…
The notion of slow entropy, both upper and lower slow entropy, was defined by Katok and Thouvenot as a more refined measure of complexity for dynamical systems, than the classical Kolmogorov-Sinai entropy. For any subexponential rate…
We study the relative entropy of the empirical probability vector with respect to the true probability vector in multinomial sampling of $k$ categories, which, when multiplied by sample size $n$, is also the log-likelihood ratio statistic.…
We propose a definition for the entropy of capacities defined on lattices. Classical capacities are monotone set functions and can be seen as a generalization of probability measures. Capacities on lattices address the general case where…
A number of recent works have sought to generalize the Kolmogorov-Sinai entropy of probability-preserving transformations to the setting of Markov operators acting on the integrable functions on a probability space $(X,\mu)$. These have…
Entropies of mixing can be derived directly from the parent distributions of extreme value theory. They correspond to pseudo-additive entropies in the case of Pareto and power function distributions, while to the Shannon entropy in the case…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
We give a characterization of Maximum Entropy/Minimum Relative Entropy inference by providing two `strong entropy concentration' theorems. These theorems unify and generalize Jaynes' `concentration phenomenon' and Van Campenhout and Cover's…
The concept of entropy, firstly introduced in information theory, rapidly became popular in many applied sciences via Shannon's formula to measure the degree of heterogeneity among observations. A rather recent research field aims at…
We partially generalize Peters' formula on modules over the group ring ${\mathbb F} \Gamma$ for a given finite field ${\mathbb F}$ and a sofic group $\Gamma$. It is also discussed that how the values of entropy are related to the zero…
The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…
We show that the essential properties of entropy (monotonicity, additivity and subadditivity) are consequences of entropy being a monoidal natural transformation from the under category functor $-/\mathsf{LProb}_{\rho}$ (where…
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of…
We introduce a new measure of interdependence among the components of a random vector along the main diagonal of the vector copula, i.e. along the line $u_{1}=\ldots=u_{J}$, for $\left(u_{1},\ldots,u_{J}\right)\in\left[0,1\right]^{J}$. Our…
A common meadow is an enrichment of a field with a partial division operation that is made total by assuming that division by zero takes the a default value, a special element $\bot$ adjoined to the field. To a common meadow of real numbers…