Related papers: Entropy and long-range correlations in random symb…
We develop an instanton technique for calculations of correlation functions characterizing statistical behavior of the elastic string in disordered media and apply the proposed approach to correlations of string free energies corresponding…
Herding is a deterministic algorithm used to generate data points that can be regarded as random samples satisfying input moment conditions. The algorithm is based on the complex behavior of a high-dimensional dynamical system and is…
The translation of written language has been known since the 3rd century BC; however, its necessity has become increasingly common in the information age. Today, many translators exist, based on encoder-decoder deep architectures,…
Entropy estimation of information sources is highly non trivial for symbol sequences with strong long-range correlations. The rabbit sequence, related to the symbolic dynamics of the nonlinear circle map at the critical point as well as the…
Network theory has often disregarded many-body relationships, solely focusing on pairwise interactions: neglecting them, however, can lead to misleading representations of complex systems. Hypergraphs represent a suitable framework for…
In a recent Letter [1] a framework for estimating entropy was introduced and applied to one-dimensional and two-dimensional systems. In this Comment we show that the method is not well suited for estimating entropy in bidimensional systems…
We introduce a graph-signal generalisation of Sample Entropy, denoted SampEn$_{G}$, to quantify irregularity of graph signals on a continuous state space, complementing existing methods on symbolic dynamics. Our approach replaces the…
Contextual entropy is a psycholinguistic measure capturing the anticipated difficulty of processing a word just before it is encountered. Recent studies have tested for entropy-related effects as a potential complement to well-known effects…
We present an analytical description of the distribution of diagonal lines in Recurrence Plots (RPs) for white noise and chaotic systems, and find that the latter one is linked to the correlation entropy. Further we identify two scaling…
A numerical method is proposed to remove the quenched randomness from a data sequence of numbers. A polymer chain of beads is introduced with both a hard core interaction and an appropriate energy associated with the data sequence. The…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
In many fields, such as language acquisition, neuropsychology of language, the study of aging, and historical linguistics, corpora are used for estimating the diversity of grammatical structures that are produced during a period by an…
Written language is complex. A written text can be considered an attempt to convey a meaningful message which ends up being constrained by language rules, context dependence and highly redundant in its use of resources. Despite all these…
We present a methodology to characterize synchronization in time series based on symbolic representations. A symbol is linked to a sequence of numbers through the rank-order of its values. A representation of a time series results after…
We afford the problem of counting the blocks of a given length made with symbols drawn from an alphabet and relate this number to Fibonacci-like recurrent relations. The recurrence polynomia allows to calculate the limit ratio of two…
Language models (LMs) are trained on billions of tokens in an attempt to recover the true language distribution. Still, vanilla random sampling from LMs yields low quality generations. Decoding algorithms attempt to restrict the LM…
Expressive text encoders such as RNNs and Transformer Networks have been at the center of NLP models in recent work. Most of the effort has focused on sentence-level tasks, capturing the dependencies between words in a single sentence, or…
Differential symbolic entropy, a measure for nonlinear dynamics complexity, is proposed in our contribution. With flexible controlling parameter, the chaotic deterministic measure takes advantage of local nonlinear dynamical information…
Quantifying the similarity between symbolic sequences is a traditional problem in Information Theory which requires comparing the frequencies of symbols in different sequences. In numerous modern applications, ranging from DNA over music to…
We prove existence of asymptotic entropy of random walks on regular languages over a finite alphabet and we give formulas for it. Furthermore, we show that the entropy varies real-analytically in terms of probability measures of constant…