Related papers: Learning Physics from Data: a Thermodynamic Interp…
We propose a thermodynamics-based learning strategy for non-equilibrium evolution equations based on Onsager's variational principle, which allows to write such PDEs in terms of two potentials: the free energy and the dissipation potential.…
Biological systems have to build models from their sensory data that allow them to efficiently process previously unseen inputs. Here, we study a neural network learning a linearly separable rule using examples provided by a teacher. We…
Video representation learning has recently attracted attention in computer vision due to its applications for activity and scene forecasting or vision-based planning and control. Video prediction models often learn a latent representation…
Concise, accurate descriptions of physical systems through their conserved quantities abound in the natural sciences. In data science, however, current research often focuses on regression problems, without routinely incorporating…
Modeling complex spatiotemporal dynamical systems, such as the reaction-diffusion processes, have largely relied on partial differential equations (PDEs). However, due to insufficient prior knowledge on some under-explored dynamical…
Irreversible processes accomplished in a fixed time involve nonlinearly coupled flows of matter, energy, and information. Here, using entropy production as an example, we show how thermodynamic uncertainty relations and speed limits on…
Typical deep learning approaches to modeling high-dimensional data often result in complex models that do not easily reveal a new understanding of the data. Research in the deep learning field is very actively pursuing new methods to…
We develop a thermodynamic framework for modeling innovation adoption and abandonment dynamics using statistical mechanics. Starting from a mathematical model for an adoption distribution that fits empirically obtained date, we construct a…
Machine learning offers an intriguing alternative to first-principles analysis for discovering new physics from experimental data. However, to date, purely data-driven methods have only proven successful in uncovering physical laws…
The training algorithms for AI systems all introduce far-from-equilibrium dynamical processes, and understanding the irreversibility of these algorithms is a fundamental step towards understanding the learning dynamics of modern AI systems.…
We expand upon a natural analogy between Bayesian statistics and statistical physics in which sample size corresponds to inverse temperature. This analogy motivates the definition of two novel statistical quantities: a learning capacity and…
Adaptive physical and biological systems continually process fluctuating information from their environments. When the environment is nonstationary, inference itself becomes a nonequilibrium process with thermodynamic cost. We analyse a…
Stochastic chains represent a wide and key variety of phenomena in many branches of science within the context of Information Theory and Thermodynamics. They are typically approached by a sequence of independent events or by a memoryless…
Machine learning is the dominant approach to artificial intelligence, through which computers learn from data and experience. In the framework of supervised learning, a necessity for a computer to learn from data accurately and efficiently…
One of the most exciting applications of artificial intelligence (AI) is automated scientific discovery based on previously amassed data, coupled with restrictions provided by known physical principles, including symmetries and conservation…
In the previous papers (Kui\'{c} et al. in Found Phys 42:319-339, 2012; Kui\'{c} in arXiv:1506.02622, 2015), it was demonstrated that applying the principle of maximum information entropy by maximizing the conditional information entropy,…
Hamiltonian mechanics is an effective tool to represent many physical processes with concise yet well-generalized mathematical expressions. A well-modeled Hamiltonian makes it easy for researchers to analyze and forecast many related…
We propose a method for learning dynamical systems from high-dimensional empirical data that combines variational autoencoders and (spatio-)temporal attention within a framework designed to enforce certain scientifically-motivated…
An isolated system of interacting quantum particles is described by a Hamiltonian operator. Hamiltonian models underpin the study and analysis of physical and chemical processes throughout science and industry, so it is crucial they are…
This paper employs a formal connection of machine learning with thermodynamics to characterize the quality of learnt representations for transfer learning. We discuss how information-theoretic functional such as rate, distortion and…