Related papers: On the Rate of Information Loss in Memoryless Syst…
A central concept in the connection between physics and information theory is entropy, which represents the amount of information extracted from the system by the observer performing measurements in an experiment. Indeed, Jaynes' principle…
The mathematical model of a linear system with the short memory about own stochastic behavior is proposed. It is assumed that the system is under a continual influence of independent stochastic impulses. In a short memory approximation the…
We consider the probability of data loss, or equivalently, the reliability function for an erasure coded distributed data storage system under worst case conditions. Data loss in an erasure coded system depends on probability distributions…
New concepts from nonequilibrium thermodynamics are used to show that Landauer's principle can be understood in terms of time asymmetry in the dynamical randomness generated by the physical process of the erasure of digital information. In…
Exploiting previous results on Markovian dynamics and fluctuation theorems, we study the consequences of memory effects on single realizations of nonequilibrium processes within an open system approach. The entropy production along single…
We study a system in which two-state Markov sources send status updates to a common receiver over a slotted ALOHA random access channel. We characterize the performance of the system in terms of state estimation entropy (SEE), which…
Information functionals allow to quantify the degree of randomness of a given probability distribution, either absolutely (through min/max entropy principles) or relative to a prescribed reference one. Our primary aim is to analyze the…
We introduce the problem of releasing sensitive data under differential privacy when the privacy level is subject to change over time. Existing work assumes that privacy level is determined by the system designer as a fixed value before…
In this paper we present the concept of description of random processes in complex systems with the discrete time. It involves the description of kinetics of discrete processes by means of the chain of finite-difference non-Markov equations…
In this paper, we present a detailed framework to analyze the evolution of the random topology of a time-varying wireless network via the information theoretic notion of entropy rate. We consider a propagation channel varying over time with…
The problem of lossless data compression with side information available to both the encoder and the decoder is considered. The finite-blocklength fundamental limits of the best achievable performance are defined, in two different versions…
We consider a system model of a general finite-state machine (ratchet) that simultaneously interacts with three kinds of reservoirs: a heat reservoir, a work reservoir, and an information reservoir, the latter being taken to be a running…
We present a theory and accompanying importance sampling method for computing rate constants in spatially inhomogenious systems. Using the relationship between rate constants and path space partition functions, we illustrate that the…
It is widely accepted that information erasure entails heat dissipation. Here we analyze asymmetric memory states to show that this energy cost can be shuffled around to any step in a write-erase cycle and need not accompany the logically…
We derive a fundamental upper bound on the rate at which a device can process information (i.e., the number of logical operations per unit time), arising from quantum mechanics and general relativity. In Planck units a device of volume V…
I formulate an entropy-rate maximization problem at the observable level for stochastic processes observed through an information-reducing observation map. For a visible stationary law, the map determines an observational fiber of hidden…
Typically, real-world stochastic processes are not easy to analyze. In this work we study the representation of any stochastic process as a memoryless innovation process triggering a dynamic system. We show that such a representation is…
Landauer's Principle states that the energy cost of information processing must exceed the product of the temperature and the change in Shannon entropy of the information-bearing degrees of freedom. However, this lower bound is achievable…
Quantum memory effects can be qualitatively understood as a consequence of an environment-to-system backflow of information. Here, we analyze and compare how this concept is interpreted and implemented in different approaches to quantum…
In biological systems, information is frequently transferred with Poisson like spike processes (shot noise) modulated in time by information-carrying signals. How then to quantify information transfer for the output for such nonstationary…