Related papers: Combinatorial Entropies and Statistics
Exponential random graph models have attracted significant research attention over the past decades. These models are maximum-entropy ensembles under the constraints that the expected values of a set of graph observables are equal to given…
We explore connections between homotopy type theory and information theory through homotopy cardinality. We define probability types and random variable types, prove that homotopy cardinality respects dependent sums under truncation and…
On a compact group the Haar probability measure plays the role of uniform distribution. The entropy and rate distortion theory for this uniform distribution is studied. New results and simplified proofs on convergence of convolutions on…
Configurational entropy is an important factor in the free energy change of many macromolecular recognition and binding processes, and has been intensively studied. Despite great progresses that have been made, the global sampling remains…
We consider an isolated system in an arbitrary state and provide a general formulation using first principles for an additive and non-negative statistical quantity that is shown to reproduce the equilibrium thermodynamic entropy of the…
In the study of economic networks, econometric approaches interpret the traditional Gravity Model specification as the expected link weight coming from a probability distribution whose functional form can be chosen arbitrarily, while…
We define an entropy based on a chosen governing probability distribution. If a certain kind of measurements follow such a distribution it also gives us a suitable scale to study it with. This scale will appear as a link function that is…
For the purpose of causal inference we employ a stochastic model of the data generating process, utilizing individual propensity probabilities for the treatment, and also individual and counterfactual prognosis probabilities for the…
It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight…
Many classical theorems in combinatorics establish the emergence of substructures within sufficiently large collections of objects. Well-known examples are Ramsey's theorem on monochromatic subgraphs and the Erd\H{o}s-Rado sunflower lemma.…
We demonstrate that Shannon's information entropy and the thermodynamic entropy of Boltzmann and Gibbs are quantitatively equivalent for real condensed-matter systems. By interpreting atomic configurations as information sources, we compute…
Maximum Entropy is a powerful concept that entails a sharp separation between relevant and irrelevant variables. It is typically invoked in inference, once an assumption is made on what the relevant variables are, in order to estimate a…
High-order phenomena play crucial roles in many systems of interest, but their analysis is often highly nontrivial. There is a rich literature providing a number of alternative information-theoretic quantities capturing high-order…
The well-known Heisenberg--Robertson uncertainty relation for a pair of noncommuting observables, is expressed in terms of the product of variances and the commutator among the operators, computed for the quantum state of a system.…
Asymptotically entropy of chaotic systems increases linearly and the sensitivity to initial conditions is exponential with time: these two behaviors are related. Such relationship is the analogous of and under specific conditions has been…
Boltzmann's struggle with a derivation of the Second Law of Thermodynamics is sketched. So is his first derivation of the connection between entropy and probability in 1877. Planck's derivation and quantum mechanical modifications of…
We consistently formalize the probabilistic description of multipartite joint measurements performed on systems of any nature. This allows us: (1) to specify in probabilistic terms the difference between nonsignaling, the Einstein-…
Increasing the number $N$ of elements of a system typically makes the entropy to increase. The question arises on {\it what particular entropic form} we have in mind and {\it how it increases} with $N$. Thermodynamically speaking it makes…
Entropy serves as a central observable which indicates uncertainty in many chemical, thermodynamical, biological and ecological systems, and the principle of the maximum entropy (MaxEnt) is widely supported in natural science. Recently,…
The Boltzmann-Gibbs celebrated entropy $S_{BG}=-k\sum_ip_i \ln p_i$ is {\it concave} (with regard to all probability distributions $\{p_i\}$) and {\it stable} (under arbitrarily small deformations of any given probability distribution). It…