Related papers: Information Loss in Static Nonlinearities
We explore the application of a new theory of Semantic Information to the well-motivated problem of a resource foraging agent. Semantic information is defined as the subset of correlations, measured via the transfer entropy, between agent…
We analyze the effect of lossy compression in the processing of sensor signals that must be used to detect anomalous events in the system under observation. The intuitive relationship between the quality loss at higher compression and the…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
One of the goals of probabilistic inference is to decide whether an empirically observed distribution is compatible with a candidate Bayesian network. However, Bayesian networks with hidden variables give rise to highly non-trivial…
Information flow analysis is a powerful technique for reasoning about the sensitive information exposed by a program during its execution. While past work has proposed information theoretic metrics (e.g., Shannon entropy, min-entropy,…
In order to find out the limiting speed of solving a specific problem using computer, this essay provides a method based on information entropy. The relationship between the minimum computational complexity and information entropy change is…
We find the relation between reliability and entropy production in a realistic model of electronic memory (low-power MOS-based SRAM) where logical values are encoded as metastable non-equilibrium states. We employ large deviations…
The present paper is devoted to investigation of the entropy reduction and entanglement-assisted classical capacity (information gain) of continuous variable quantum measurements. These quantities are computed explicitly for multimode…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
Entropy governs molecular self-assembly, phase transitions, and material stability, yet remains challenging to quantify and directly control in molecular systems. Here, we demonstrate that the computable information density (CID), a data…
Maximum entropy principle identifies forces conjugated to observables and the thermodynamic relations between them, independent upon their underlying mechanistic details. For data about state distributions or transition statistics, the…
This paper deals with the data-driven synthesis of dissipative linear systems in discrete time. We collect finitely many noisy data samples with which we synthesise a controller that makes all systems that explain the data dissipative with…
The principle of entropy increase is not only the basis of statistical mechanics, but also closely related to the irreversibility of time, the origin of life, chaos and turbulence. In this paper, we first discuss the dynamic system…
In this paper we give a thorough presentation of a model proposed by Tononi et al. for modeling \emph{integrated information}, i.e. how much information is generated in a system transitioning from one state to the next one by the causal…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
The paper deals with measures of nonlinearity. In state estimation, they are utilized i) to select a suitable state estimation algorithm by assessing the nonlinearity of a system model, ii) to adapt the estimation algorithm structure or…
It is well known that semantic and structural features of the generated images emerge at different times during the reverse dynamics of diffusion, a phenomenon that has been connected to physical phase transitions in magnets and other…
Information theory and the framework of information dynamics have been used to provide tools to characterise complex systems. In particular, we are interested in quantifying information storage, information modification and information…
Shannon's information entropy measures of the uncertainty of an event's outcome. If learning about a system reflects a decrease in uncertainty, then a plausible intuition is that learning should be accompanied by a decrease in the entropy…
This paper proposes that the mathematical relationship between an entropy distribution and its limit offers some new insight into system performance. This relationship is used to quantify variation among the entities of a system, where…