Related papers: Learning entropy production via neural networks
Entropy is ubiquitous in machine learning, but it is in general intractable to compute the entropy of the distribution of an arbitrary continuous random variable. In this paper, we propose the amortized residual denoising autoencoder…
The ability to continuously learn remains elusive for deep learning models. Unlike humans, models cannot accumulate knowledge in their weights when learning new tasks, mainly due to an excess of plasticity and the low incentive to reuse…
Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…
In this paper, we consider the state estimation problem for nonlinear stochastic discrete-time systems. We combine Lyapunov's method in control theory and deep reinforcement learning to design the state estimator. We theoretically prove the…
The thermodynamic uncertainty relation originally proven for systems driven into a non-equilibrium steady state (NESS) allows one to infer the total entropy production rate by observing any current in the system. This kind of inference…
Accurate precipitation estimation is critical for flood forecasting, water resource management, and disaster preparedness. Satellite products provide global hourly coverage but contain systematic biases; ground-based gauges are accurate at…
Entropy production plays a fundamental role in the study of non-equilibrium systems by offering a quantitative handle on the degree of time-reversal symmetry breaking. It depends crucially on the degree of freedom considered as well as on…
The probability distribution of the total entropy production in the non-equilibrium steady state follows a symmetry relation called the fluctuation theorem. When a certain part of the system is masked or hidden, it is difficult to infer the…
The minimum entropy production principle provides an approximative variational characterization of close-to-equilibrium stationary states, both for macroscopic systems and for stochastic models. Analyzing the fluctuations of the empirical…
A positive rate of entropy production at steady state is a distinctive feature of truly non-equilibrium processes. Exact results, while being often limited to simple models, offer a unique opportunity to explore the thermodynamic features…
Unlike in the traditional statistical modeling for which a user typically hand-specify a prior, Neural Processes (NPs) implicitly define a broad class of stochastic processes with neural networks. Given a data stream, NP learns a stochastic…
Entropy in nonequilibrium statistical mechanics is investigated theoretically so as to extend the well-established equilibrium framework to open nonequilibrium systems. We first derive a microscopic expression of nonequilibrium entropy for…
Learning stochastic functions from partially observed context-target pairs requires models that are expressive, uncertainty-aware, and strongly conditioned on inputs. Neural Diffusion Processes (NDPs) improve expressivity with denoising…
We study the multiple definitions of the entropy production for discrete-time Markov processes in single systems and composite systems. These definitions have been studied in single systems, but less so in composite systems. With a clear…
Neural symbolic processing aims to combine the generalization of logical learning approaches and the performance of neural networks. The Neural Theorem Proving (NTP) model by Rocktaschel et al (2017) learns embeddings for concepts and…
In the paper, we introduce the maximum entropy estimator based on 2-dimensional empirical distribution of the observation sequence of hidden Markov model , when the sample size is big: in that case computing the maximum likelihood estimator…
Entropy estimation is a fundamental problem in information theory that has applications in various fields, including physics, biology, and computer science. Estimating the entropy of discrete sequences can be challenging due to limited data…
Using fluorescence spectroscopy we directly measure entropy production of a single two-level system realized experimentally as an optically driven defect center in diamond. We exploit a recent suggestion to define entropy on the level of a…
Ensembling is now recognized as an effective approach for increasing the predictive performance and calibration of deep networks. We introduce a new approach, Parameter Ensembling by Perturbation (PEP), that constructs an ensemble of…
Neuro-encoded expression programming(NEEP) that aims to offer a novel continuous representation of combinatorial encoding for genetic programming methods is proposed in this paper. Genetic programming with linear representation uses…