Related papers: An entropic characterization of long memory statio…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
Random matrices whose entries come from a stationary Gaussian process are studied. The limiting behavior of the eigenvalues as the size of the matrix goes to infinity is the main subject of interest in this work. It is shown that the…
A family of self-similar and translation-invariant random sup-measures with long-range dependence are investigated. They are shown to arise as the limit of the empirical random sup-measure of a stationary heavy-tailed process, inspired by…
We analyze phase transitions in the conditional entropy of a sequence caused by a change in the conditional variables. Such transitions happen, for example, when training to learn the parameters of a system, since the transition from the…
In this paper we give explicit examples of power-law correlated stationary Markovian processes y(t) where the stationary pdf shows tails which are gaussian or exponential. These processes are obtained by simply performing a coordinate…
We consider stationary ergodic processes indexed by $\mathbb Z$ or $\mathbb Z^n$ whose finite dimensional marginals have laws which are absolutely continuous with respect to Lebesgue measure. We define an entropy theory for these continuous…
Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building…
Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a COUNT associated with…
Transfer entropy is used to establish a measure of causal relationships between two variables. Symbolic transfer entropy, as an estimation method for transfer entropy, is widely applied due to its robustness against non-stationarity. This…
Complex systems are found in most branches of science. It is still argued how to best quantify their complexity and to what end. One prominent measure of complexity (the statistical complexity) has an operational meaning in terms of the…
Both marginal and dependence features must be described when modelling the extremes of a stationary time series. There are standard approaches to marginal modelling, but long- and short-range dependence of extremes may both appear. In…
Maximum Entropy (MaxEnt) reinforcement learning is a powerful learning paradigm which seeks to maximize return under entropy regularization. However, action entropy does not necessarily coincide with state entropy, e.g., when multiple…
The concept of entropy in statistical physics is related to the existence of irreversible macroscopic processes. In this work, we explore a recently introduced entropy formula for a class of stochastic processes with more than one absorbing…
Many years ago B.S. Pitskel observed that the metric entropy of the shift transformation in the sample space of a stationary random process $X=\{X_n,\,n\in \mathbb Z\}$ with a countable number of states is equal to the conditional entropy…
Machine learning is usually defined in behaviourist terms, where external validation is the primary mechanism of learning. In this paper, I argue for a more holistic interpretation in which finding more probable, efficient and abstract…
When monitoring the dynamics of stochastic systems, such as interacting particles agitated by thermal noise, disentangling deterministic forces from Brownian motion is challenging. Indeed, we show that there is an information-theoretic…
We present the results of a simulation study into the properties of 12 different estimators of the Hurst parameter, $H$, or the fractional integration parameter, $d$, in long memory time series. We compare and contrast their performance on…
Stationary stochastic processes (SPs) are a key component of many probabilistic models, such as those for off-the-grid spatio-temporal data. They enable the statistical symmetry of underlying physical phenomena to be leveraged, thereby…
The Hurst exponent is the simplest numerical summary of self-similar long-range dependent stochastic processes. We consider the estimation of Hurst exponent in long-range dependent curve time series. Our estimation method begins by…
The rate of entropy production provides a useful quantitative measure of a non-equilibrium system and estimating it directly from time-series data from experiments is highly desirable. Several approaches have been considered for stationary…