Related papers: Toward a Physical Theory of Intelligence
In noisy physical reservoirs, the classical information-processing capacity $C_{\mathrm{ip}}$ quantifies how well a linear readout can realize tasks measurable from the input history, yet $C_{\mathrm{ip}}$ can be far smaller than the…
This book presents a methodology and philosophy of empirical science based on large scale lossless data compression. In this view a theory is scientific if it can be used to build a data compression program, and it is valuable if it can…
Humans learn adaptively and efficiently throughout their lives. However, incrementally learning tasks causes artificial neural networks to overwrite relevant information learned about older tasks, resulting in 'Catastrophic Forgetting'.…
Neural operators have demonstrated considerable effectiveness in accelerating the solution of time-dependent partial differential equations (PDEs) by directly learning governing physical laws from data. However, for PDEs governed by…
Landauer's principle bounds the heat generated by logical operations, but in practice the thermodynamic cost of computation is dominated by the control systems that implement logic. CMOS gates dissipate energy far above the Landauer bound,…
Quantum Landauer's principle provides a fundamental lower bound for energy dissipation occurred with information erasure in the quantum regime. While most studies have related the entropy reduction incorporated with the erasure to the lower…
In this paper we shall relate computational complexity to the principle of natural selection. We shall do this by giving a philosophical account of complexity versus universality. It seems sustainable to equate universal systems to complex…
The human brain is the substrate for human intelligence. By simulating the human brain, artificial intelligence builds computational models that have learning capabilities and perform intelligent tasks approaching the human level. Deep…
Conventional computing has many sources of heat dissipation, but one of these--the Landauer limit--poses a fundamental lower bound of 1 bit of entropy per bit erased. 'Reversible Computing' avoids this source of dissipation, but is…
This thesis presents three different results in quantum information theory. The first result addresses the theoretical foundations of quantum metrology. The Heisenberg limit considered as the ultimate limit in quantum metrology sets a lower…
Framing computation as the transformation of metastable memories, we explore its fundamental thermodynamic limits. The true power of information follows from a novel decomposition of nonequilibrium free energy derived here, which provides a…
Multi-instance data, in which each object (bag) contains a collection of instances, are widespread in machine learning, computer vision, bioinformatics, signal processing, and social sciences. We present a maximum entropy (ME) framework for…
We investigate the behavior of a Bose-Einstein Condensate (BEC) under the influence of a central barrier as the particle number trends towards the thermodynamic limit. In order to perform these studies, we present a novel method which is…
Within an inherently classical perspective, there is always an unavoidable energy cost associated with the information deletion and this common lore is at the heart of the Landauer's conjecture that does not impose, per se, any relevant…
We propose a new way of investigating phase transitions in the context of information theory. We use an information-entropic measure of spatial complexity known as configurational entropy (CE) to quantify both the storage and exchange of…
The classic Landauer bound can be lowered when erasure errors are permitted. Here we point out that continuous phase transitions characterized by an order parameter can also be viewed as information erasure by resetting a certain number of…
One of the outstanding challenges to information processing is the eloquent suppression of energy consumption in execution of logic operations. Landauer principle sets an energy constraint in deletion of a classical bit of information.…
Computation, if treated as a set of physical processes that act on information represented by states of matter, encompasses biological systems, digital systems, and other constructs, and may be a fundamental measure of living systems. The…
To explain conceptual gap between classical/quantum and other, hypothetical descriptions of world, several principles has been proposed. So far, all these principles have not explicitly included the uncertainty relation. Here we introduce…
Modern AI systems achieve remarkable capabilities at the cost of substantial energy consumption. To connect intelligence to physical efficiency, we propose two complementary bits-per-joule metrics under explicit accounting conventions: (1)…