Related papers: Using the Memories of Multiscale Machines to Chara…
The presence of smart objects is increasingly widespread and their ecosystem, also known as Internet of Things, is relevant in many different application scenarios. The huge amount of temporally annotated data produced by these smart…
We show why the amount of information communicated between the past and future--the excess entropy--is not in general the amount of information stored in the present--the statistical complexity. This is a puzzle, and a long-standing one,…
Modern scientific instruments produce vast amounts of data, which can overwhelm the processing ability of computer systems. Lossy compression of data is an intriguing solution, but comes with its own drawbacks, such as potential signal…
For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one…
The dynamics of symbolic systems, such as multidimensional subshifts of finite type or cellular automata, are known to be closely related to computability theory. In particular, the appropriate tools to describe and classify topological…
Multivariate information theory provides a general and principled framework for understanding how the components of a complex system are connected. Existing analyses are coarse in nature -- built up from characterizations of discrete…
In this paper, we present a novel method for computing the relative entropy as well as the expected relative entropy using an MCMC chain. The relative entropy from information theory can be used to quantify differences in posterior…
The concept of entropy in nonequilibrium macroscopic systems is investigated in the light of an extended equation of motion for the density matrix obtained in a previous study. It is found that a time-dependent information entropy can be…
Contemporary lossy image and video coding standards rely on transform coding, the process through which pixels are mapped to an alternative representation to facilitate efficient data compression. Despite impressive performance of…
Information theoretic quantities are extremely useful in discovering relationships between two or more data sets. One popular method---particularly for continuous systems---for estimating these quantities is the nearest neighbour…
We investigate the memory properties of discrete sequences built upon a finite number of states. We find that the block entropy can reliably determine the memory for systems modeled as Markov chains of arbitrary finite order. Further, we…
We propose a sample-based, sequential method to abstract a (potentially black-box) dynamical system with a sequence of memory-dependent Markov chains of increasing size. We show that this approximation allows to alleviating a correlation…
A hidden Markov process is a well known concept in information theory and is used for a vast range of applications such as speech recognition and error correction. We bridge between two disciplines, experimental physics and advanced…
Algebraic representations of time series are symbolic representations whose symbols belong to a finite group. Precisely, the framework of the present paper is the analysis of coupled time series in algebraic representations and, more…
We introduce a method for quantifying the inherent unpredictability of a continuous-valued time series via an extension of the differential Shannon entropy rate. Our extension, the specific entropy rate, quantifies the amount of predictive…
Computers are deterministic dynamical systems (CHAOS 19:033124, 2009). Among other things, that implies that one should be able to use deterministic forecast rules to predict their behavior. That statement is sometimes-but not always-true.…
In this work we introduce a method for estimating entropy rate and entropy production rate from finite symbolic time series. From the point of view of statistics, estimating entropy from a finite series can be interpreted as a problem of…
Humans are adept at uncovering abstract associations in the world around them, yet the underlying mechanisms remain poorly understood. Intuitively, learning the higher-order structure of statistical relationships should involve complex…
When an expert operates a perilous dynamic system, ideal constraint information is tacitly contained in their demonstrated trajectories and controls. The likelihood of these demonstrations can be computed, given the system dynamics and task…
We apply information theoretic entropies of coordinate and velocity distributions in quantum mechanics for the description of the strong field ionization process. The approach is based on the properties of the entropies used in the…