Related papers: Measuring the Complexity of Continuous Distributio…
Written language is complex. A written text can be considered an attempt to convey a meaningful message which ends up being constrained by language rules, context dependence and highly redundant in its use of resources. Despite all these…
Permutation entropy measures the complexity of deterministic time series via a data symbolic quantization consisting of rank vectors called ordinal patterns or just permutations. The reasons for the increasing popularity of this entropy in…
Probability distributions can be read as simple expressions of information. Each continuous probability distribution describes how information changes with magnitude. Once one learns to read a probability distribution as a measurement scale…
A novel heuristic approach is proposed here for time series data analysis, dubbed Generalized weighted permutation entropy, which amalgamates and generalizes beyond their original scope two well established data analysis methods:…
Complex systems are characterised by a tight, nontrivial interplay of their constituents, which gives rise to a multi-scale spectrum of emergent properties. In this scenario, it is practically and conceptually difficult to identify those…
A generalization of a distribution increases the flexibility particularly in studying of a phenomenon and its properties. Many generalizations of continuous univariate distributions are available in literature. In this study, an…
Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide…
We introduce and discuss the notion of monotonicity for the complexity measures of general probability distributions, patterned after the resource theory of quantum entanglement. Then, we explore whether this property is satisfied by the…
Information entropy is applied to the analysis of time series generated by dynamical systems. Complexity of a temporal or spatio-temporal signal is defined as the difference between the sum of entropies of the local linear regions of the…
We review briefly the concepts underlying complex systems and probability distributions. The later are often taken as the first quantitative characteristics of complex systems, allowing one to detect the possible occurrence of regularities…
A central issue of the science of complex systems is the quantitative characterization of complexity. In the present work we address this issue by resorting to information geometry. Actually we propose a constructive way to associate to a -…
The Shannon entropy, and related quantities such as mutual information, can be used to quantify uncertainty and relevance. However, in practice, it can be difficult to compute these quantities for arbitrary probability distributions,…
Systematic generalization remains challenging for current language models, which are known to be both sensitive to semantically similar permutations of the input and to struggle with known concepts presented in novel contexts. Although…
We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, R\'enyi, generalized R\'enyi, Tsallis, Sharma-Mittal, and modified Shannon entropy,…
In this paper, we identify a class of absolutely continuous probability distributions, and show that the differential entropy is uniformly convergent over this space under the metric of total variation distance. One of the advantages of…
Measuring the complexity of high-dimensional data in physical systems becomes a critical factor in determining the information and quality of the systems. However, traditional metrics, such as Lyapunov exponent, fractal dimension, and…
We show that the natural scaling of measurement for a particular problem defines the most likely probability distribution of observations taken from that measurement scale. Our approach extends the method of maximum entropy to use…
Entropy is a central concept in physics, but can be challenging to calculate even for systems that are easily simulated. This is exacerbated out of equilibrium, where generally little is known about the distribution characterizing simulated…
We study the build up of complexity on the example of 1 kg matter in different forms. We start on the simplest example of ideal gases, and then continue with more complex chemical, biological, life and social and technical structures. We…
Preserving biodiversity and ecosystem stability is a challenge that can be pursued through modern statistical mechanics modeling. Here we introduce a variational maximum entropy-based algorithm to evaluate the entropy in a minimal ecosystem…