Related papers: Maximum Entropy Principle, Equal Probability a Pri…
Maximum Entropy is a powerful concept that entails a sharp separation between relevant and irrelevant variables. It is typically invoked in inference, once an assumption is made on what the relevant variables are, in order to estimate a…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
The foundations of the Boltzmann-Gibbs (BG) distributions for describing equilibrium statistical mechanics of systems are examined. Broadly, they fall into: (i) probabilistic paaroaches based on the principle of equal a priori probability…
We investigate the overdamped stochastic dynamics of a particle in an asymptotically flat external potential field, in contact with a thermal bath. For an infinite system size, the particles may escape the force field and diffuse freely at…
Traditionally, the Method of (Shannon-Kullback's) Relative Entropy Maximization (REM) is considered with linear moment constraints. In this work, the method is studied under frequency moment constraints which are non-linear in…
For a wide range of entropy measures, easy calculation of equilibria is possible using a principle of Game Theoretical Equilibrium related to Jaynes Maximum Entropy Principle. This follows previous work of the author and relates to works of…
The issue of the thermodynamics of a system of distinguishable particles is discussed in this paper. In constructing the statistical mechanics of distinguishable particles from the definition of Boltzmann entropy, it is found that the…
We study the continuity of the maximum-entropy inference map for two observables in finite dimensions. We prove that the continuity is equivalent to the strong continuity of the set-valued inverse numerical range map. This gives a…
I present an unbiased method of mapping particles to distribution functions and vice versa. This method alone defines the canonical formulation of statistical mechanics, since it can be used to derive the principle of maximum entropy in…
We propose a general methodology for performing statistical inference within a `rare-events regime' that was recently suggested by Wagner, Viswanath and Kulkarni. Our approach allows one to easily establish consistent estimators for a very…
Exponential families form the backbone of modern statistics and machine learning, but textbooks seldom derive them from first principles in an accessible way. Although minimal sufficiency and the principle of maximum entropy, originating in…
The entropy definition in the microcanonical ensemble is revisited. We propose a novel definition for the microcanonical entropy that resolve the debate on the correct definition of the microcanonical entropy. In particular we show that…
We discuss how maximum entropy methods may be applied to the reconstruction of Markov processes underlying empirical time series and compare this approach to usual frequency sampling. It is shown that, at least in low dimension, there…
Counting ad infinitum is the holographic observable to a statistical dynamics with finite states under independent repeated sampling. Entropy provides the infinitesimal probability for an observed frequency $\hat{\boldsymbol{\nu}}$ w.r.t. a…
We revisit the classical problem of inverting dimension-reducing linear mappings using the maximum entropy (MaxEnt) criterion. In the literature, solutions are problem-dependent, inconsistent, and use different entropy measures. We propose…
During the MaxEnt 2002 workshop in Moscow, Idaho, Tony Vignaux asked again a few simple questions about using Maximum Entropy or Bayesian approaches for the famous Dice problems which have been analyzed many times through this workshop and…
Supplement 1 to GUM (GUM-S1) recommends the use of maximum entropy principle (MaxEnt) in determining the probability distribution of a quantity having specified properties, e.g., specified central moments. When we only know the mean value…
A theory for non-equilibrium systems is derived from a maximum entropy approach similar in spirit to the equilibrium theory given by Gibbs. Requiring Hamilton's principle of stationary action to be satisfied on average during a trajectory,…
We derive Bose-Einstein statistics and Fermi-Dirac statistics by Principle of Maximum Entropy applied to two families of entropy functions different from the Boltzmann-Gibbs-Shannon entropy. These entropy functions are identified with…
We examine the {combinatorial} or {probabilistic} definition ("Boltzmann's principle") of the entropy or cross-entropy function $H \propto \ln \mathbb{W}$ or $D \propto - \ln \mathbb{P}$, where $\mathbb{W}$ is the statistical weight and…