Related papers: Some Results on the Information Loss in Dynamical …
In this work, we investigate the relation between the concept of ``information rate'', an information geometric method for measuring the speed of the time evolution of the statistical states of a stochastic process, and stochastic…
We prove that statistical information theoretic quantities, such as information entropy, cannot generally be interrelated with the lower limit of energy dissipation during information erasure. We also point out that, in deterministic and…
We consider the problem of learning the dynamics of a linear system when one has access to data generated by an auxiliary system that shares similar (but not identical) dynamics, in addition to data from the true system. We use a weighted…
We analyze a power distribution line with high penetration of distributed generation and strong variations of power consumption and generation levels. In the presence of uncertainty the statistical description of the system is required to…
The concept of entropy in nonequilibrium macroscopic systems is investigated in the light of an extended equation of motion for the density matrix obtained in a previous study. It is found that a time-dependent information entropy can be…
We review the physical foundations of Landauer's Principle, which relates the loss of information from a computational process to an increase in thermodynamic entropy. Despite the long history of the Principle, its fundamental rationale and…
We provide a stochastic extension of the Baez-Fritz-Leinster characterization of the Shannon information loss associated with a measure-preserving function. This recovers the conditional entropy and a closely related information-theoretic…
A basic task of information processing is information transfer (flow). Here we study a pair of Brownian particles each coupled to a thermal bath at temperature $T_1$ and $T_2$, respectively. The information flow in such a system is defined…
In this paper we prove estimates on the behaviour of the Kolmogorov-Sinai entropy relative to a partition for randomly perturbed dynamical systems. Our estimates use the entropy for the unperturbed system and are obtained using the notion…
Feed-forward deep neural networks have been used extensively in various machine learning applications. Developing a precise understanding of the underling behavior of neural networks is crucial for their efficient deployment. In this paper,…
Determining the reachable set for a given nonlinear control system is crucial for system control and planning. However, computing such a set is impossible if the system's dynamics are not fully known. This paper is motivated by a scenario…
In the context of state estimation under communication constraints, several notions of dynamical entropy play a fundamental role, among them: topological entropy and restoration entropy. In this paper, we present a theorem which…
This paper investigates approximation-theoretic aspects of the in-context learning capability of the transformers in representing a family of noisy linear dynamical systems. Our first theoretical result establishes an upper bound on the…
Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…
In order to find out the limiting speed of solving a specific problem using computer, this essay provides a method based on information entropy. The relationship between the minimum computational complexity and information entropy change is…
For a closed-loop control system with a digital channel between the sensor and the controller, the notion of invariance entropy quantifies the smallest average rate of information above which a given compact subset of the state space can be…
In this paper, we wish to investigate the dynamics of information transfer in evolutionary dynamics. We use information theoretic tools to track how much information an evolving population has obtained and managed to retain about different…
We derive a fundamental upper bound on the rate at which a device can process information (i.e., the number of logical operations per unit time), arising from quantum mechanics and general relativity. In Planck units a device of volume V…
We study classes of dynamical systems that can be obtained by constructing recursive networks with monotone Boolean functions. Stack filters in nonlinear signal processing are special cases of such systems. We show an analytical connection…
We consider the problem of quantifying information flow in interactive systems, modelled as finite-state transducers in the style of Goguen and Meseguer. Our main result is that if the system is deterministic then the information flow is…