Related papers: Occam's Razor Cuts Away the Maximum Entropy Princi…
In this tutorial we review the essential arguments behing entropic inference. We focus on the epistemological notion of information and its relation to the Bayesian beliefs of rational agents. The problem of updating from a prior to a…
The selection of an equilibrium state by maximising the entropy of a system, subject to certain constraints, is often powerfully motivated as an exercise in logical inference, a procedure where conclusions are reached on the basis of…
Recommendations based on behavioral data may be faced with ambiguous statistical evidence. We consider the case of association rules, relevant e.g.~for query and product recommendations. For example: Suppose that a customer belongs to…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…
The thermodynamic definition of entropy can be extended to nonequilibrium systems based on its relation to information. To apply this definition in practice requires access to the physical system's microstates, which may be prohibitively…
We show that the principle of entropy increase may be exactly founded on a few axioms valid not only for quantum and classical statistics, but also for a wide range of statistical processes.
We construct the generalized entropy optimized by a given arbitrary statistical distribution with a finite linear expectation value of a random quantity of interest. This offers, via the maximum entropy principle, a unified basis for a…
We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and…
This article applies the principle of Occam's Razor to non-parametric model building of statistical data, by finding a model with the minimal number of bits, leading to an exceptionally effective regularization method for probability…
Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies…
A new axiomatic characterization with a minimum of conditions for entropy as a function on the set of states in quantum mechanics is presented. Traditionally unspoken assumptions are unveiled and replaced by proven consequences of the…
The classical Maximum-Entropy Principle (MEP) based on Shannon entropy is widely used to construct least-biased probability distributions from partial information. However, the Shore-Johnson axioms that single out the Shannon functional…
Given a knowledge base KB containing first-order and statistical facts, we consider a principled method, called the random-worlds method, for computing a degree of belief that some formula Phi holds given KB. If we are reasoning about a…
The fundamentals of the Maximum Entropy principle as a rule for assigning and updating probabilities are revisited. The Shannon-Jaynes relative entropy is vindicated as the optimal criterion for use with an updating rule. A constructive…
We consider relaxation of an isolated system to the equilibrium using detailed balance condition and Onsager's fluctuation approximation. There is a small deviation from the equilibrium in two parameters. For this system, explicit…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
Entropy serves as a central observable which indicates uncertainty in many chemical, thermodynamical, biological and ecological systems, and the principle of the maximum entropy (MaxEnt) is widely supported in natural science. Recently,…
This paper's first aim is to prove a modernized Occam's razor beyond a reasonable doubt. To summarize the main argument in one sentence: If we consider all possible, intelligible, scientific models of ever-higher complexity, democratically,…
The maximum entropy principle advocates to evaluate events' probabilities using a distribution that maximizes entropy among those that satisfy certain expectations' constraints. Such principle can be generalized for arbitrary decision…
Far-from-equilibrium thermodynamics underpins the emergence of life, but how has been a long-outstanding puzzle. Best candidate theories based on the maximum entropy production principle could not be unequivocally proven, in part due to…