Related papers: Toward a Physical Theory of Intelligence
We review the physical foundations of Landauer's Principle, which relates the loss of information from a computational process to an increase in thermodynamic entropy. Despite the long history of the Principle, its fundamental rationale and…
Landauer's principle introduces a symmetry between computational and physical processes: erasure of information, a logically irreversible operation, must be underlain by an irreversible transformation dissipating energy. Monitoring micro-…
Context-aware compression techniques have gained increasing attention as model sizes continue to grow, introducing computational bottlenecks that hinder efficient deployment. A structured encoding approach was proposed to selectively…
The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits,…
Is the dynamical evolution of physical systems objectively a manifestation of information processing by the universe? We find that an affirmative answer has important consequences for the measurement problem. In particular, we calculate the…
We investigate fundamental connections between thermodynamics and quantum information theory. First, we show that the operational framework of thermal operations is nonequivalent to the framework of Gibbs-preserving maps, and we comment on…
Reducing the energy inefficiency of conventional CMOS-based computing devices -- which rely on logically irreversible gates to process information -- remains both a fundamental engineering challenge and a practical social challenge of…
Unsupervised learning plays an important role in many fields, such as artificial intelligence, machine learning, and neuroscience. Compared to static data, methods for extracting low-dimensional structure for dynamic data are lagging. We…
According to the Landauer principle, any logically irreversible process accompanies entropy production, which results in heat dissipation in the environment. Erasing of information, one of the primary logically irreversible processes, has a…
A dominant paradigm in visual intelligence treats semantics as a static property of latent representations, assuming that meaning can be discovered through geometric proximity in high dimensional embedding spaces. In this work, we argue…
Landauer's principle provides a perspective on the physical meaning of information as well as on the minimum working cost of information processing. Whereas most studies have related the decrease in entropy during a computationally…
Landauer's Principle states that the energy cost of information processing must exceed the product of the temperature and the change in Shannon entropy of the information-bearing degrees of freedom. However, this lower bound is achievable…
What is intelligence? We argue for a structural-dynamical account rooted in a topological closure law: \emph{the boundary of a boundary vanishes} ($\partial^2=0$). This principle forces transient fragments to cancel while closed cycles…
Thermodynamics and information have intricate interrelations. Often thermodynamics is considered to be the logical premise to justify that information is physical - through Landauer's principle -, thereby also linking information and…
Landauer's Principle that information loss from a computation implies entropy increase can be rigorously proved from mathematical physics. However, carefully examining its detailed formulation reveals that the traditional identification of…
Continual learning in artificial neural networks is fundamentally limited by the stability--plasticity dilemma: systems that retain prior knowledge tend to resist acquiring new knowledge, and vice versa. Existing approaches, most notably…
The Landauer principle states that any logically irreversible information processing must be accompanied by dissipation into the environment. In this study, we investigate the heat dissipation associated with finite-time information erasure…
We formulate a geometric framework in which physical laws emerge from restricted access to microscopic information. Measurement constraints are modeled as a gauge symmetry acting on density operators, inducing a gauge-reduced space of…
This paper presents evidence for the idea that much of artificial intelligence, human perception and cognition, mainstream computing, and mathematics, may be understood as compression of information via the matching and unification of…
Information processing typically occurs via the composition of modular units, such as universal logic gates. The benefit of modular information processing, in contrast to globally integrated information processing, is that complex global…