相关论文: Maximum entropy, fluctuations and priors
Given a probability density $P({\bf x}|{\boldsymbol \lambda})$, where $\bf x$ represents continuous degrees of freedom and $\lambda$ a set of parameters, it is possible to construct a general identity relating expectations of observable…
Fluctuation theorems establish that thermodynamic processes at the microscale can occasionally result in negative entropy production. At the microscale, another distinct possibility becomes more likely: processes in which no entropy is…
We point out a limitation of the mutual information neural estimation (MINE) where the network fails to learn at the initial training phase, leading to slow convergence in the number of training iterations. To solve this problem, we propose…
Tempering is a popular tool in Bayesian computation, being used to transform a posterior distribution $p_1$ into a reference distribution $p_0$ that is more easily approximated. Several algorithms exist that start by approximating $p_0$ and…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
Transfer entropy measures directed information flow in time series, and it has become a fundamental quantity in applications spanning neuroscience, finance, and complex systems analysis. However, existing estimation methods suffer from the…
It is known that the Maximum relative Entropy (MrE) method can be used to both update and approximate probability distributions functions in statistical inference problems. In this manuscript, we apply the MrE method to infer magnetic…
Inferring the value of a property of a large stochastic system is a difficult task when the number of samples is insufficient to reliably estimate the probability distribution. The Bayesian estimator of the property of interest requires the…
Analytic continuation of numerical data obtained in imaginary time or frequency has become an essential part of many branches of quantum computational physics. It is, however, an ill-conditioned procedure and thus a hard numerical problem.…
Heat fluctuations over a time \tau in a non-equilibrium stationary state and in a transient state are studied for a simple system with deterministic and stochastic components: a Brownian particle dragged through a fluid by a harmonic…
Thermodynamic entropy is determined by a heat measurement through the Clausius equality. The entropy then formalizes a fundamental limitation of operations by the second law of thermodynamics. The entropy is also expressed as the Shannon…
Stochastic thermodynamics as reviewed here systematically provides a framework for extending the notions of classical thermodynamics like work, heat and entropy production to the level of individual trajectories of well-defined…
Mixture models are widely used in Bayesian statistics and machine learning, in particular in computational biology, natural language processing and many other fields. Variational inference, a technique for approximating intractable…
Many physicists think that the maximum entropy formalism is a straightforward application of Bayesian statistical ideas to statistical mechanics. Some even say that statistical mechanics is just the general Bayesian logic of inductive…
Finite heat reservoir capacity and temperature fluctuations lead to modification of the well known canonical exponential weight factor. Requiring that the corrections least depend on the one-particle energy, we derive a deformed entropy,…
We explore a method of statistical estimation called Maximum Entropy on the Mean (MEM) which is based on an information-driven criterion that quantifies the compliance of a given point with a reference prior probability measure. At the core…
The entropy per particle in most Monte-Carlo simulations is size dependent due to correlated energy fluctuations. Guided by nanothermodynamics, we find a constraint for the Ising model that enhances the fluctuations and lowers the free…
We develop a maximum relative entropy formalism to generate optimal approximations to probability distributions. The central results consist in (a) justifying the use of relative entropy as the uniquely natural criterion to select a…
Prompted by the realisation that the statistical entropy of an ideal gas in the micro-canonical ensemble should not fluctuate or change over time, the meaning of the H-theorem is re-interpreted from the perspective of information theory in…
This paper addresses the question of the fluctuations of the empirical entropy of a chain of infinite order. We assume that the chain takes values on a finite alphabet and loses memory exponentially fast. We consider two possible…