Related papers: Free Energy Minimization: A Unified Framework for …
We present a new outlook on the climate system thermodynamics, studying some of its macroscopic properties in terms of the 1st and 2nd laws of thermodynamics. We review and clarify the notion of efficiency of the climate system by…
Physics and economics are two disciplines that share the common challenge of linking microscopic and macroscopic behaviors. However, while physics is based on collective dynamics, economics is based on individual choices. This conceptual…
This study formulates the ideal efficiency of light-powered systems in the most general form, based on the first principle of energy-entropy flow analysis under the condition of zero entropy generation within the system. A unified formula…
In this paper we develop a generalized formalism for equilibrium thermodynamic systems when an information is shared between the system and the reservoir. The information results in a correction to the entropy of the system. This extension…
Using an alternative notion of entropy introduced by Datta, the max-entropy, we present a new simplified framework to study the minimizers of the specific free energy for random fields which are weakly dependent in the sense of Lewis,…
We study the sign problem in lattice field theory with a $\theta$ term. We apply the maximum entropy method (MEM) to flattening phenomenon of the free energy density $f(\theta)$, which originates from the sign problem. In our previous…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible while constrained to match empirically estimated feature expectations. However, in many real-world…
Energy-based learning is a powerful learning paradigm that encapsulates various discriminative and generative approaches. An energy-based model (EBM) is typically formed of inner-model(s) that learn a combination of the different features…
A growing body of research in continual learning focuses on the catastrophic forgetting problem. While many attempts have been made to alleviate this problem, the majority of the methods assume a single model in the continual learning…
This paper presents an information-theoretic framework for unifying active learning problems: level set estimation (LSE), Bayesian optimization (BO), and their generalized variant. We first introduce a novel active learning criterion that…
Adaptive physical and biological systems continually process fluctuating information from their environments. When the environment is nonstationary, inference itself becomes a nonequilibrium process with thermodynamic cost. We analyse a…
Generative models such as denoising diffusion models are quickly advancing their ability to approximate highly complex data distributions. They are also increasingly leveraged in scientific machine learning, where samples from the implied…
Maximum likelihood estimation (MLE) of latent variable models is often recast as the minimization of a free energy functional over an extended space of parameters and probability distributions. This perspective was recently combined with…
A central challenge for intelligent agents in an uncertain world is striking the right balance between utility maximization and resource use, not only for external movement but also for internal computation. Existing theories of control…
Batteryless systems frequently face power failures, requiring extra runtime buffers to maintain inference progress and leaving only a memory space for storing ultra-tiny deep neural networks (DNNs). Besides, making these models responsive…
Reducing the fuel consumption within a power network is crucial to enhance the overall system efficiency and minimize operating costs. Fuel consumption minimization can be achieved through different optimization techniques where the output…
We develop a continuous-time reinforcement learning framework for a class of singular stochastic control problems without entropy regularization. The optimal singular control is characterized as the optimal singular control law, which is a…
A thermodynamic formalism describing the efficiency of information learning is proposed, which is applicable for stochastic thermodynamic systems with multiple internal degree of freedom. The learning rate, entropy production rate (EPR),…
Econophysics, is based on the premise that some ideas and methods from physics can be applied to economic situations. We intend to show in this paper how a physics concept such as entropy can be applied to an economic problem. In so doing,…
We study the energy minimization problem in low-level vision tasks from a novel perspective. We replace the heuristic regularization term with a learnable subspace constraint, and preserve the data term to exploit domain knowledge derived…