Related papers: Information thermodynamics: from physics to neuros…
Integrated Information Theory (IIT) is an audacious attempt to pin down the abstract, phenomenological experiences of consciousness into a rigorous, mathematical framework. We show that IIT's stance in regards to neuronal noise is…
How the human brain processes information during different cognitive tasks is one of the greatest questions in contemporary neuroscience. Understanding the statistical properties of brain signals during specific activities is one promising…
An approach to implementing variational Bayesian inference in biological systems is considered, under which the thermodynamic free energy of a system directly encodes its variational free energy. In the case of the brain, this assumption…
Although information theory resolves inconsistencies (known under the form of famous enigmas) of the traditional approach of thermostatistics, its place in the corresponding literature is not what it deserves. This article supports the idea…
In this paper, we analyze the relationship between entropy and information in the context of the mixing process of two identical ideal gases. We will argue that entropy has a special information-based feature that is enfolded in the…
Scientific discovery can be framed as a thermodynamic process in which an agent invests physical work to acquire information about an environment under a finite work budget. Using established results about the thermodynamics of computing,…
We focus attention on some particular thermodynamic relations (PTR). Using information theory concepts we show that, for a reversible process, microscopic considerations related to these PTR make the concomitant informational contents of…
The activity of a neural network is defined by patterns of spiking and silence from the individual neurons. Because spikes are (relatively) sparse, patterns of activity with increasing numbers of spikes are less probable, but with more…
We illustrate in terms familiar to modern day science students that: (i) an uncertainty slope mechanism underlies the usefulness of temperature via its reciprocal, which is incidentally around 42 [nats/eV] at the freezing point of water;…
Dense Associative Memory networks (DenseAMs) unify several popular paradigms in Artificial Intelligence (AI), such as Hopfield Networks, transformers, and diffusion models, while casting their computational properties into the language of…
Information based thermodynamic logic is revisited. It consists of two parts: Part A applies the modern theory of probability in which an arbitrary convex function \phi is employed as an analytic "device" to express information as…
This tutorial investigates the convergence of statistical mechanics and learning theory, elucidating the potential enhancements in machine learning methodologies through the integration of foundational principles from physics. The tutorial…
A central concept in the connection between physics and information theory is entropy, which represents the amount of information extracted from the system by the observer performing measurements in an experiment. Indeed, Jaynes' principle…
Information plays a pivotal role in the thermodynamics of nonequilibrium processes with feedback. However, much remains to be learned about the nature of information fluctuations in small scale devices and their relation with fluctuations…
How do organisms recognize their environment by acquiring knowledge about the world, and what actions do they take based on this knowledge? This article examines hypotheses about organisms' adaptation to the environment from machine…
These are notes for a set of 7 two-hour lectures given at the 2010 Summer School on Quantitative Evolutionary and Comparative Genomics at OIST, Okinawa, Japan. The emphasis is on understanding how biological systems process information. We…
To understand under which conditions thermodynamics emerges from the microscopic dynamics is the ultimate goal of statistical mechanics. Despite the fact that the theory is more than 100 years old, we are still discussing its foundations…
One of the greatest research challenges of this century is to understand the neural basis for how behavior emerges in brain-body-environment systems. To this end, research has flourished along several directions but have predominantly…
We prove that statistical information theoretic quantities, such as information entropy, cannot generally be interrelated with the lower limit of energy dissipation during information erasure. We also point out that, in deterministic and…
The particle-in-cell numerical method of plasma physics balances a trade-off between computational cost and intrinsic noise. Inference on data produced by these simulations generally consists of binning the data to recover the particle…