Related papers: Entropy from Machine Learning
The statistical picture of the solution space for a binary perceptron is studied. The binary perceptron learns a random classification of input random patterns by a set of binary synaptic weights. The learning of this network is difficult…
Entropy and free-energy estimation are key in thermodynamic characterization of simulated systems ranging from spin models through polymers, colloids, protein structure, and drug-design. Current techniques suffer from being model specific,…
Configurational entropy is an important factor in the free energy change of many macromolecular recognition and binding processes, and has been intensively studied. Despite great progresses that have been made, the global sampling remains…
The paper proposes a new message passing algorithm for cycle-free factor graphs. The proposed "entropy message passing" (EMP) algorithm may be viewed as sum-product message passing over the entropy semiring, which has previously appeared in…
The goal of these lecture notes is to review the problem of free energy minimization as a unified framework underlying the definition of maximum entropy modelling, generalized Bayesian inference, learning with latent variables, statistical…
The montecarlo method, which is quite commonly used to solve maximum entropy problems in statistical physics, can actually be used to solve inverse problems in a much wider context. The probability distribution which maximizes entropy can…
Optimisation problems in science and engineering typically involve finding the ground state (i.e. the minimum energy configuration) of a cost function with respect to many variables. If the variables are corrupted by noise then this…
Efficient and accurate algorithm for partition function, free energy and thermal entropy calculations is of great significance in statistical physics and quantum many-body physics. Here we present an unbiased but low-technical-barrier…
We devise a hierarchy of computational algorithms to enumerate the microstates of a system comprising N independent, distinguishable particles. An important challenge is to cope with integers that increase exponentially with system size,…
We calculate the density of states of a binary Lennard-Jones glass using a recently proposed Monte Carlo algorithm. Unlike traditional molecular simulation approaches, the algorithm samples distinct configurations according to…
To make clear several issues relating with the thermodynamics of computations, we perform a simulation of a binary device using a Langevin equation. Based on our numerical results, we consider how to estimate thermodynamic entropy of…
We study the statistical physics of the classical Ising model in the so-called $\alpha$-R\'enyi ensemble, a finite-temperature thermal state approximation that minimizes a modified free energy based on the $\alpha$-R\'enyi entropy. We begin…
The entropy of a binary symmetric Hidden Markov Process is calculated as an expansion in the noise parameter epsilon. We map the problem onto a one-dimensional Ising model in a large field of random signs and calculate the expansion…
We use Monte Carlo and genetic algorithms to train neural-network feedback-control protocols for simulated fluctuating nanosystems. These protocols convert the information obtained by the feedback process into heat or work, allowing the…
The entropy change that occurs upon mixing two fluids has remained an intriguing topic since the dawn of statistical mechanics. In this work, we generalize the grand-isobaric ensemble to mixtures, and develop a Monte Carlo algorithm for the…
In order to gain a deeper understanding of complex systems and infer key information using minimal data, I classify all configurations based on classical probability, starting from the dimensions of energy and different categories of…
Boltzmann sampling based on Metropolis algorithm has been extensively used for simulating a canonical ensemble. An estimate of a mechanical property, like energy, of an equilibrium system, can be made by averaging over a large number…
We explore a supervised machine learning approach to estimate the entanglement entropy of multi-qubit systems from few experimental samples. We put a particular focus on estimating both aleatoric and epistemic uncertainty of the network's…
Algorithmic entropy can be seen as a special case of entropy as studied in statistical mechanics. This viewpoint allows us to apply many techniques developed for use in thermodynamics to the subject of algorithmic information theory. In…
Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…