Related papers: Maximum entropy principle and power-law tailed dis…
Gauss' law of error is generalized in Tsallis statistics such as multifractal systems, in which Tsallis entropy plays an essential role instead of Shannon entropy. For the generalization, we apply the new multiplication operation determined…
Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies…
Macroscopic mechanical properties of polymers are determined by their microscopic molecular chain distribution. Due to randomness of these molecular chains, probability theory has been used to find their micro-states and energy…
We examine the {combinatorial} or {probabilistic} definition ("Boltzmann's principle") of the entropy or cross-entropy function $H \propto \ln \mathbb{W}$ or $D \propto - \ln \mathbb{P}$, where $\mathbb{W}$ is the statistical weight and…
Using statistical thermodynamics, we derive a general expression of the stationary probability distribution for thermodynamic systems driven out of equilibrium by several thermodynamic forces. The local equilibrium is defined by imposing…
The Boltzmann entropy $S^{(B)}$ is true in the case of equal probability of all microstates of a system. In the opposite case it should be averaged over all microstates that gives rise to the Boltzmann--Shannon entropy (BSE). Maximum…
Entropy and relative or cross entropy measures are two very fundamental concepts in information theory and are also widely used for statistical inference across disciplines. The related optimization problems, in particular the maximization…
Within its range of applicability, the Boltzmann equation seems unique in its capacity to accurately describe the transition from almost any initial state to a self-equilibrated thermal state. Using information-theoretic methods to rephrase…
The Large Deviation Principle (LDP) and the Central Limit Theorem (CLT) are central pillars of probability theory. While their formulations are established under the i.i.d. assumption, the probabilistic foundation for power-law…
Dissipative processes cause collisionless plasmas in many systems to develop nonthermal particle distributions with broad power-law tails. The prevalence of power-law energy distributions in space/astrophysical observations and kinetic…
A finite dimensional-system whose physics is governed by a Gaussian distribution can be regarded as a subsystem of an infinite dimensional-underlying system described by a uniform distribution on the (infinite dimensional) sphere. In turn,…
Many physicists think that the maximum entropy formalism is a straightforward application of Bayesian statistical ideas to statistical mechanics. Some even say that statistical mechanics is just the general Bayesian logic of inductive…
Behavior of condensed matter systems deviating from the standard equilibrium conditions is discussed. Statistical properties of coupled dynamic-stochastic systems are studied within a combination of the maximum information principle and the…
Maximum entropy principle identifies forces conjugated to observables and the thermodynamic relations between them, independent upon their underlying mechanistic details. For data about state distributions or transition statistics, the…
The theory of probability shows that, as the fraction $X_n/Y\to 0$, the conditional probability for $X_n$, given $X_n+Y \in h_{\delta}:=[h, h+\delta]$, has a limit law $f_{X_n}(x)e^{-\psi_n(h_\delta)x}$, where $\psi_n(h_\delta) $ equals to…
The classic central limit theorem and $\alpha$-stable distributions play a key role in probability theory, and also in Boltzmann-Gibbs (BG) statistical mechanics. They both concern the paradigmatic case of probabilistic independence of the…
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
The word-frequency distribution provides the fundamental building blocks that generate discourse in language. It is well known, from empirical evidence, that the word-frequency distribution of almost any text is described by Zipf's law, at…
Many natural and artificial systems whose range of interaction is long enough are known to exhibit (quasi)stationary states that defy the standard, Boltzmann-Gibbs statistical mechanical prescriptions. For handling such anomalous systems…