Related papers: Three Perspectives on Complexity $-$ Entropy, Comp…
A simple, intuitive approach to the assessment of probabilistic inferences is introduced. The Shannon information metrics are translated to the probability domain. The translation shows that the negative logarithmic score and the geometric…
In this paper, we introduce the notion of recurrence entropy in the context of nonlinear control systems. A set is said to be ($\tau$-)recurrent if every trajectory that starts in the set returns to it (within at most $\tau$ units of time).…
The paper reviews two prominent approaches for the measurement of technological complexity: the method of reflection and the assessment of technologies' combinatorial difficulty. It discusses their central underlying assumptions and…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
The aim of this paper is to introduce the Lempel-Ziv permutation complexity vs permutation entropy plane as a tool to analyze time series of different nature. This two quantities make use of the Bandt and Pompe representation to quantify…
Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…
We apply renormalized entropy as a complexity measure to the logistic and sine-circle maps. In the case of logistic map, renormalized entropy decreases (increases) until the accumulation point (after the accumulation point up to the most…
In 2002, in a seminal article, Christoph Bandt and Bernd Pompe proposed a new methodology for the analysis of complex time series, now known as Ordinal Analysis. The ordinal methodology is based on the computation of symbols (known as…
Process mining is a research area focusing on the design of algorithms that can automatically provide insights into business processes. Among the most popular algorithms are those for automated process discovery, which have the ultimate…
Observing how long a dynamical system takes to return to some state is one of the most simple ways to model and quantify its dynamics from data series. This work proposes two formulas to estimate the KS entropy and a lower bound of it, a…
Physics concepts have often been borrowed and independently developed by other fields of science. In this perspective a significant example is that of entropy in Information Theory. The aim of this paper is to provide a short and…
We trace the history of empirical entropy, touching briefly on its relation to Markov processes, normal numbers, Shannon entropy, the Chomsky hierarchy, Kolmogorov complexity, Ziv-Lempel compression, de Bruijn sequences and stochastic…
A novel heuristic approach is proposed here for time series data analysis, dubbed Generalized weighted permutation entropy, which amalgamates and generalizes beyond their original scope two well established data analysis methods:…
Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…
Understanding how humans perceive visual complexity is a key area of study in visual cognition. Previous approaches to modeling visual complexity assessments have often resulted in intricate, difficult-to-interpret algorithms that employ…
The complex behavior of many systems in nature requires the application of robust methodologies capable of identifying changes in their dynamics. In the case of time series (which are sensed values of a system during a time interval),…
A constraint satisfaction problem (CSP) is a computational problem where the input consists of a finite set of variables and a finite set of constraints, and where the task is to decide whether there exists a satisfying assignment of values…
We study neural network compressibility by using singular learning theory to extend the minimum description length (MDL) principle to singular models like neural networks. Through extensive experiments on the Pythia suite with quantization,…
The notion of metric entropy dimension is introduced to measure the complexity of entropy zero dynamical systems. For measure preserving systems, we define entropy dimension via the dimension of entropy generating sequences. This…
Compressed Counting (CC), based on maximally skewed stable random projections, was recently proposed for estimating the p-th frequency moments of data streams. The case p->1 is extremely useful for estimating Shannon entropy of data…