Related papers: Maximum entropy, fluctuations and priors
A well-known result across information theory, machine learning, and statistical physics shows that the maximum entropy distribution under a mean constraint has an exponential form called the Gibbs-Boltzmann distribution. This is used for…
A novel formalism, called H-theory, is applied to the problem of statistical equilibrium of a hierarchical complex system with multiple time and length scales. In this approach, the system is formally treated as being composed of a small…
Thermodynamics is usually developed starting from entropy and the maximum entropy principle. We investigate here to what extent one can replace entropy with relative entropy which has several advantages, for example in the context of local…
Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with…
In the general process of eliminating dynamic variables in Markovian models, there exists a difference in the irreversible entropy production between the original and reduced dynamics. We call this difference the hidden entropy production,…
We propose a mixedness quantifier based on entropy fluctuations. It provides information about the degree of mixedness either for finite dimensional and infinite dimensional Hilbert spaces. It may be used to determine the reduction of the…
The formalism used in describing the thermodynamics of abrupt (or first-order) phase transitions is reviewed as an application of maximum entropy inference. In this treatment, we show that the concepts of transition temperature, latent heat…
The kinematics and dynamics of deterministic physical systems have been a foundation of our understanding of the world since Galileo and Newton. For real systems, however, uncertainty is largely present via external forces such as friction…
We demonstrate how information in the form of observable data and moment constraints are introduced into the method of Maximum relative Entropy (ME). A general example of updating with data and moments is shown. A specific econometric…
A broad set of sufficient conditions that guarantees the existence of the maximum entropy (maxent) distribution consistent with specified bounds on certain generalized moments is derived. Most results in the literature are either focused on…
The issue of discrete probability estimation for samples of small size is addressed in this study. The maximum likelihood method often suffers over-fitting when insufficient data is available. Although the Bayesian approach can avoid…
Temperature of a finite-sized system fluctuates due to the thermal fluctuations. However, a systematic mathematical framework for measuring or estimating the temperature is still underdeveloped. Here, we incorporate the estimation theory in…
The montecarlo method, which is quite commonly used to solve maximum entropy problems in statistical physics, can actually be used to solve inverse problems in a much wider context. The probability distribution which maximizes entropy can…
Jaynes' maximum entropy (MaxEnt) principle was recently used to give a conditional, local derivation of the ``maximum entropy production'' (MEP) principle, which states that a flow system with fixed flow(s) or gradient(s) will converge to a…
We consider the problem of estimating the population probability distribution given a finite set of multivariate samples, using the maximum entropy approach. In strict keeping with Jaynes' original definition, our precise formulation of the…
The ongoing unprecedented exponential explosion of available computing power, has radically transformed the methods of statistical inference. What used to be a small minority of statisticians advocating for the use of priors and a strict…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…
The Maximum Entropy (MaxEnt) technique is applied to the derivation of the Gaussian Dispersion Plume Model as well as to more complex transport phenomena such as the one-dimensional advection equation, the one-dimensional diffusion…
We study entropy production (EP) in processes involving repeated quantum measurements of finite quantum systems. Adopting a dynamical system approach, we develop a thermodynamic formalism for the EP and study fine aspects of irreversibility…
We present a maximum entropy approach to analyze the internal dynamics of a small system in contact with a large bath e.g. a solute-solvent system. For the small solute, the fluctuations around the mean values of observables are not…