Related papers: Learning rate matrix and information-thermodynamic…
We show state-of-the-art word representation learning methods maximize an objective function that is a lower bound on the mutual information between different parts of a word sequence (i.e., a sentence). Our formulation provides an…
Trade-off relations place fundamental limits on the operations that physical systems can perform. This Letter presents a trade-off relation that bounds the correlation function, which measures the relationship between a system's current and…
We investigate the parameter space of transformer models trained on protein sequence data using a statistical mechanics framework, sampling the loss landscape at varying temperatures by Langevin dynamics to characterize the low-loss…
Human learners have the natural ability to use knowledge gained in one setting for learning in a different but related setting. This ability to transfer knowledge from one task to another is essential for effective learning. In this paper,…
Thermodynamics and information have intricate interrelations. Often thermodynamics is considered to be the logical premise to justify that information is physical - through Landauer's principle -, thereby also linking information and…
Ultrahigh lattice thermal conductivity materials hold great importance since they play a critical role in the thermal management of electronic and optical devices. Models using machine learning can search for materials with outstanding…
Quantum information scrambling refers to the loss of local recoverability of quantum information, which has found widespread attention from high energy physics to quantum computing. In the present analysis we propose a possible starting…
Data shuffling is one of the fundamental building blocks for distributed learning algorithms, that increases the statistical gain for each step of the learning process. In each iteration, different shuffled data points are assigned by a…
Active matter generates order or patterns through nonequilibrium dynamics. An open research challenge is to determine how efficiently a nonequilibrium self-organising system can convert consumed energy into macroscopic order. We study an…
The cost of information processing in physical systems calls for a trade-off between performance and energetic expenditure. Here we formulate and study a computation-dissipation bottleneck in mesoscopic systems used as input-output devices.…
The capacity to integrate information is a prominent feature of biological and cognitive systems. Integrated Information Theory (IIT) provides a mathematical approach to quantify the level of integration in a system, yet its computational…
Non-Hermitian systems offer new platforms for unusual physical properties that can be flexibly manipulated by redistribution of the real and imaginary parts of refractive indices, whose presence breaks conventional wave propagation…
The optimal quantum measurements for estimating individual parameters might be incompatible with each other so that they cannot be jointly performed. The tradeoff between the estimation precision for different parameters can be…
We develop a novel framework for costly information acquisition in which a decision-maker learns about an unobserved state by choosing a signal distribution, with the cost of information determined by the distribution of noise in the…
We study nonequilibrium thermodynamics of complex information flows induced by interactions between multiple fluctuating systems. Characterizing nonequilibrium dynamics by causal networks (i.e., Bayesian networks), we obtain novel…
Information flow or information transfer is an important concept in dynamical systems which has applications in a wide variety of scientific disciplines. In this study, we show that a rigorous formalism can be established in the context of…
Using information entropy formalism, we consider a one-dimensional system with heat flux and extend the meaning of equilibrium variables to non equilibrium scenarios when classical local equilibrium approach is not applicable; this is…
This paper investigates the nonlinear dynamics and phase transitions in power packet network connected with routers, conceptualized as macroscopic information-ratchets. In the emerging paradigm of cyber-physical energy systems, the…
Information measures are often used to assess the efficacy of neural networks, and learning rules can be derived through optimization procedures on such measures. In biological neural networks, computation is restricted by the amount of…
This paper analyzes a dynamic interaction between a fully rational, privately informed sender and a boundedly rational, uninformed receiver with memory constraints. The sender controls the flow of information, while the receiver designs a…