相关论文: Free Energy Minimization: A Unified Framework for …

In these decades, it has been revealed that there is rich information-theoretic structure in thermodynamics of out-of-equilibrium systems in both the classical and quantum regimes. This has led to the fruitful interplay among statistical…

Szeliski et al. published an influential study in 2006 on energy minimization methods for Markov Random Fields (MRF). This study provided valuable insights in choosing the best optimization technique for certain classes of problems. While…

In this second article, we show a simple use of the Ignorance as defined in "Jaynes & Shannon's Constrained Ignorance and Surprise". By giving an example about the journey of a person, we believe to show some simple, obvious but…

Bayesian inference is a widely used statistical method. The free energy and generalization loss, which are used to estimate the accuracy of Bayesian inference, are known to be small in singular models that do not have a unique optimal…

This paper presents a continuous, information-theoretic extension of the Free Energy Principle through the concept of Markov blanket density, i.e., a scalar field that quantifies the degree of conditional independence between internal and…

In this chapter, we discuss recent advances and new opportunities through methods of machine learning for the field of classical density functional theory, dealing with the equilibrium properties of thermal nano- and micro-particle systems…

We present a general holistic theory for the organization of complex networks, both human-engineered and naturally-evolved. Introducing concepts of value of interactions and satisfaction as generic network performance measures, we show that…

Many hallmarks of human intelligence, such as generalizing from limited experience, abstract reasoning and planning, analogical reasoning, creative problem solving, and capacity for language require the ability to consolidate experience…

An oft-cited challenge of federated learning is the presence of heterogeneity. \emph{Data heterogeneity} refers to the fact that data from different clients may follow very different distributions. \emph{System heterogeneity} refers to the…

The Variation After Projection approach is applied for the first time to the pairing hamiltonian to describe the thermodynamics of small systems with fixed particle number. The minimization of the free energy is made by a direct…

Accurate free energy representations are crucial for understanding phase dynamics in materials. We employ a scale-bridging approach to incorporate atomistic information into our free energy model by training a neural network on DFT-informed…

In this paper, we consider multi-objective reinforcement learning, which arises in many real-world problems with multiple optimization goals. We approach the problem with a max-min framework focusing on fairness among the multiple goals and…

Artificial intelligence models trained through loss minimization have demonstrated significant success, grounded in principles from fields like information theory and statistical physics. This work explores these established connections…

The success of deep learning has been due, in no small part, to the availability of large annotated datasets. Thus, a major bottleneck in current learning pipelines is the time-consuming human annotation of data. In scenarios where such…

In many real world problems, optimization decisions have to be made with limited information. The decision maker may have no a priori or posteriori data about the often nonconvex objective function except from on a limited number of points…

Training in machine learning generally consists in finding one model, whose parameters minimize a data-dependent loss. Yet, empirical work shows that ensemble learning, an approach in which multiple models are sampled, can improve…

The free energy principle, and its corollary active inference, constitute a bio-inspired theory that assumes biological agents act to remain in a restricted set of preferred states of the world, i.e., they minimize their free energy. Under…

In a previous paper, we introduced an axiomatic system for information thermodynamics, deriving an entropy function that includes both thermodynamic and information components. From this function we derived an entropic probability…

The quest to develop a general framework for thermodynamics, suitable for the regime of strong coupling and correlations between subsystems of an autonomous quantum "universe," has entailed diverging definitions for basic quantities,…

It is shown that the dual to the linear programming problem that arises in constraint-based models of metabolism can be given a thermodynamic interpretation in which the shadow prices are chemical potential analogues, and the objective is…