Related papers: On the Entropy Rate of Pattern Processes
We study the sequence entropy of rank one measure-preserving systems along subexponential sequences. We prove that the sequence entropy along a large class of sequences can be infinite using Ornstein's probabilistic constructions. Moreover,…
We introduce a simple analysis of the structural complexity of infinite-memory processes built from random samples of stationary, ergodic finite-memory component processes. Such processes are familiar from the well known multi-arm Bandit…
The entropy of an ergodic finite-alphabet process can be computed from a single typical sample path x_1^n using the entropy of the k-block empirical probability and letting k grow with $n$ roughly like log n. We further assume that the…
Causal inference seeks to identify cause-and-effect interactions in coupled systems. A recently proposed method by Liang detects causal relations by quantifying the direction and magnitude of information flow between time series. The…
Stationary reciprocal processes defined on a finite interval of the integer line can be seen as a special class of Markov random fields restricted to one dimension. Non stationary reciprocal processes have been extensively studied in the…
The rate of entropy production provides a useful quantitative measure of a non-equilibrium system and estimating it directly from time-series data from experiments is highly desirable. Several approaches have been considered for stationary…
The time evolution of complex systems usually can be described through stochastic processes. These processes are measured at finite resolution, what necessarily reduces them to finite sequences of real numbers. In order to relate these data…
We develop a theory of classical complexity. We study the relations between classical complexity and entropy, and conjecture that in an isolated system, classical absolute complexity always tends to grow, until it reaches its maximum. We…
Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide…
We study the permutation complexity of finite-state stationary stochastic processes based on a duality between values and orderings between values. First, we establish a duality between the set of all words of a fixed length and the set of…
We study the N-step binary stationary ergodic Markov chain and analyze its differential entropy. Supposing that the correlations are weak we express the conditional probability function of the chain through the pair correlation function and…
For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one…
A probabilistic approach of computing geometric rate of convergence of stochastic processes is introduced in this paper. The goal is to quantitatively compute both upper and lower bounds of the exponential rate of convergence to the…
In this paper we investigate a quantity called conditional entropy of ordinal patterns, akin to the permutation entropy. The conditional entropy of ordinal patterns describes the average diversity of the ordinal patterns succeeding a given…
Long memory or long range dependency is an important phenomenon that may arise in the analysis of time series or spatial data. Most of the definitions of long memory of a stationary process $X=\{X_1, X_2,\cdots,\}$ are based on the…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Information theory on a time-discrete setting in the framework of time series analysis is generalized to the time-continuous case. Considerations of the Roessler and Lorenz dynamics as well as the Ornstein-Uhlenbeck process yield for…
We provide an ergodic theory framework to study statistical properties of smooth sequences over the odd alphabet {1, 3}. The arithmetic nature of this alphabet yields a partition of the subshift of smooth sequences based on their local…
The logarithm of the number of Eulerian orientations, normalised by the number of vertices, is known as the residual entropy in studies of ice-type models on graphs. The spanning tree entropy depends similarly on the number of spanning…
We investigate the problem of Multiple Description (MD) coding of discrete ergodic processes. We introduce the notion of MD stationary coding, and characterize its relationship to the conventional block MD coding. In stationary coding, in…