Related papers: Combinatorial Entropies and Statistics
Boltzmann's Principle S = k ln W was repeatedly criticized by Einstein since it lacked a proper dynamical foundation in view of the thermal motion of the particles, out of which a physical system consists. This suggests, in particular, that…
To characterize strongly interacting statistical systems within a thermodynamical framework - complex systems in particular - it might be necessary to introduce generalized entropies, $S_g$. A series of such entropies have been proposed in…
In this paper we recall for physicists how it is possible, using the principle of maximization of the Boltzmann-Shannon entropy, to derive the Burr-Bingh-Maddala (burr12) double power law probability distribution function and its…
Abundance estimation from capture-recapture data is of great importance in many disciplines. Analysis of capture-recapture data is often complicated by the existence of one-inflation and heterogeneity problems. Simultaneously taking these…
Traditional statistical mechanics is constrained by the binary paradigms of identical/distinguishable and bosonic/fermionic particle statistics, leading to a fundamental logical gap in describing systems with partial distinguishability. We…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
We give a new characterization of relative entropy, also known as the Kullback-Leibler divergence. We use a number of interesting categories related to probability theory. In particular, we consider a category FinStat where an object is a…
Within its range of applicability, the Boltzmann equation seems unique in its capacity to accurately describe the transition from almost any initial state to a self-equilibrated thermal state. Using information-theoretic methods to rephrase…
The Boltzmann--Gibbs entropy is a functional on the space of probability measures. When a state space is countable, one characterization of the Boltzmann--Gibbs entropy is given by the Shannon--Khinchin axioms, which consist of continuity,…
We study which outcomes are implementable by disclosing coarse statistics of a data-generating process rather than its full distribution. Players observe data whose joint distribution is only partially known: they know the expectations of…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…
In statistical mechanics, measuring the number of available states and their probabilities, and thus the system's entropy, enables the prediction of the macroscopic properties of a physical system at equilibrium. This predictive capacity…
We apply two variations of the principle of Minimum Cross Entropy (the Kullback information measure) to fit parameterized probability density models to observed data densities. For an array beamforming problem with P incident narrowband…
We present a unifying approach to the study of entropies in Mathematics, such as measure entropy, topological entropy, algebraic entropy, set-theoretic entropy. We take into account discrete dynamical systems, that is, pairs $(X,T)$, where…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
Along recent decades, an intensive worldwide research activity is focusing both black holes and cosmos (e.g. the dark-energy phenomenon) on the basis of entropic approaches. The Boltzmann-Gibbs-based Bekenstein-Hawking entropy…
This paper proposes a novel entropy encoding technique for lossless data compression. Representing a message string by its lexicographic index in the permutations of its symbols results in a compressed version matching Shannon entropy of…
We reconsider the Boltzmann-Gibbs statistical ensemble in thermodynamics using the multinomial coefficient approach. We show that an ensemble is defined by the determination of four statistical quantities, the element probabilities $p_i$,…
A principle of hierarchical entropy maximization is proposed for generalized superstatistical systems, which are characterized by the existence of three levels of dynamics. If a generalized superstatistical system comprises a set of…