Related papers: Zero and negative energy dissipation at informatio…
Concept erasure is the task of erasing information about a concept (e.g., gender or race) from a representation set while retaining the maximum possible utility -- information from original representations. Concept erasure is useful in…
We initiate the systematic study of the energy complexity of algorithms (in addition to time and space complexity) based on Landauer's Principle in physics, which gives a lower bound on the amount of energy a system must dissipate if it…
To comply with AI and data regulations, the need to forget private or copyrighted information from trained machine learning models is increasingly important. The key challenge in unlearning is forgetting the necessary data in a timely…
Connections between information theory and thermodynamics have proven to be very useful to establish bounding limits for physical processes. Ideas such as Landauer's erasure principle and information assisted work extraction have greatly…
Quantum state engineering and quantum computation rely on information erasure procedures that, up to some fidelity, prepare a quantum object in a pure state. Such processes occur within Landauer's framework if they rely on an interaction…
While Landauer's Principle sets a lower bound for the work required for a computation, that work is recoverable for efficient computations. However, practical physical computers, such as modern digital computers or biochemical systems, are…
Quantum Landauer's principle provides a fundamental lower bound for energy dissipation occurred with information erasure in the quantum regime. While most studies have related the entropy reduction incorporated with the erasure to the lower…
The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits,…
We investigate the Einstein vacuum equations as well as the Einstein-null fluid equations describing neutrino radiation. We find new structures in gravitational waves and memory for asymptotically-flat spacetimes of slow decay. These…
We study faulty successive cancellation decoding of polar codes for the binary erasure channel. To this end, we introduce a simple erasure-based fault model and we show that, under this model, polarization does not happen, meaning that…
We extend present Shannon's static statistical information theory to dynamic processes and establish a dynamic statistical information theory. We derive the nonlinear evolution equations of dynamic information density and dynamic…
We demonstrate that the Maximum Lyapunov Exponent for computable dynamical systems is isomorphic to the maximum capacity of a noiseless, memoryless channel in a Shannon communication model. The isomorphism allows the understanding of…
Let $W$ be a binary-input memoryless symmetric (BMS) channel with Shannon capacity $I(W)$ and fix any $\alpha > 0$. We construct, for any sufficiently small $\delta > 0$, binary linear codes of block length $O(1/\delta^{2+\alpha})$ and rate…
Memoryless channels with deletion errors as defined by a stochastic channel matrix allowing for bit drop outs are considered in which transmitted bits are either independently deleted with probability $d$ or unchanged with probability…
We review the physical foundations of Landauer's Principle, which relates the loss of information from a computational process to an increase in thermodynamic entropy. Despite the long history of the Principle, its fundamental rationale and…
Landauer's principle, often regarded as the foundation of the thermodynamics of information processing, holds that any logically irreversible manipulation of information, such as the erasure of a bit or the merging of two computation paths,…
Consider communication over a channel whose probabilistic model is completely unknown vector-wise and is not assumed to be stationary. Communication over such channels is challenging because knowing the past does not indicate anything about…
We show that the way in which the Shannon entropy of sequences produced by an information source converges to the source's entropy rate can be used to monitor how an intelligent agent builds and effectively uses a predictive model of its…
Signal transduction, the information processing mechanism in biological cells, is carried out by a network of biochemical reactions. The dynamics of driven biochemical reactions can be studied in terms of nonequilibrium statistical physics.…
We briefly address Landauer's Principle and some related issues in thermal demons. We show that an error-free Turing computer works in the zero-entropy limit, which proves Landauer's derivation incorrect. To have a physical logic gate,…