Related papers: Watts-per-Intelligence Part II: Algorithmic Cataly…
The artificial intelligence industry is not an isolated economic phenomenon; it is the current physical substrate for a broader, multi-billion-year process: the evolution of an abstract intelligence on Earth. As the scale of computation…
The statistical mechanical interpretation of algorithmic information theory (AIT, for short) was introduced and developed by our former works [K. Tadaki, Local Proceedings of CiE 2008, pp.425-434, 2008] and [K. Tadaki, Proceedings of…
Landauer's Principle states that the energy cost of information processing must exceed the product of the temperature and the change in Shannon entropy of the information-bearing degrees of freedom. However, this lower bound is achievable…
We present an experiment in which a one-bit memory is constructed, using a system of a single colloidal particle trapped in a modulated double-well potential. We measure the amount of heat dissipated to erase a bit and we establish that in…
This letter exposes a tight connection between the thermodynamic efficiency of information processing and predictive inference. A generalized lower bound on dissipation is derived for partially observable information engines which are…
We develop a statistical mechanical interpretation of algorithmic information theory by introducing the notion of thermodynamic quantities, such as free energy, energy, statistical mechanical entropy, and specific heat, into algorithmic…
We present three different neural network algorithms to calculate thermodynamic properties as well as dynamic correlation functions at finite temperatures for quantum lattice models. The first method is based on purification, which allows…
The Landauer principle sets a fundamental thermodynamic constraint on the minimum amount of heat that must be dissipated to erase one logical bit of information through a quasi-statically slow protocol. For finite time information erasure,…
The study of intelligent systems explains behaviour in terms of economic rationality. This results in an optimization principle involving a function or utility, which states that the system will evolve until the configuration of maximum…
The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits,…
The energy cost of erasing quantum states depends on our knowledge of the states. We show that learning algorithms can acquire such knowledge to erase many copies of an unknown state at the optimal energy cost. This is proved by showing…
In this thesis, I investigate various aspects of one of the most fundamental questions in thermodynamics: what state transformations can quantum systems undergo while interacting with a thermal bath under specific constraints? These…
Landauer's principle bounds the heat generated by logical operations, but in practice the thermodynamic cost of computation is dominated by the control systems that implement logic. CMOS gates dissipate energy far above the Landauer bound,…
Landauer's bound is the minimum thermodynamic cost for erasing one bit of information. As this bound is achievable only for quasistatic processes, finite-time operation incurs additional energetic costs. We find a tight finite-time…
Information-processing systems that coordinate multiple agents and objectives face fundamental thermodynamic constraints. We show that solutions with maximum utility to act as coordination focal points have a much higher selection pressure…
The rapid growth of deep neural networks (DNNs) has brought increasing attention to their energy use during training and inference. Here, we establish the thermodynamic bounds on energy consumption in quasi-static analog DNNs by mapping…
Reasoning models excel at complex tasks such as coding and mathematics, yet their inference is often slow and token-inefficient. To improve the inference efficiency, post-training quantization (PTQ) usually comes with the cost of large…
Using a double-well potential as a physical memory, we study with experiments and numerical simulations the energy exchanges during erasure processes, and model quantitatively the cost of fast operation. Within the stochastic thermodynamics…
We derive a bound on the efficiency of thermal engines that can be sharper than Carnot's limit. It is a function of statistical correlations between the engine internal state and Hamiltonian, can be saturated even in finite-time cycles, and…
We develop a physics-based model for classical computation based on autonomous quantum thermal machines. These machines consist of few interacting quantum bits (qubits) connected to several environments at different temperatures. Heat flows…