Related papers: On the Entropy of a Two Step Random Fibonacci Subs…
In statistical thermodynamics the 2nd law is properly spelled out in terms of conditioned probabilities. As such it makes the statement, that `entropy increases with time' without preferring a time direction. In this paper I try to explain…
Topologically constrained genome-like polymers often double-fold into tree-like configurations. Here we calculate the exact number of tightly double-folded configurations available to a ring polymer in ideal conditions. For this purpose, we…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
A sofic approximation to a countable group is a sequence of partial actions on finite sets that asymptotically approximates the action of the group on itself by left-translations. A group is sofic if it admits a sofic approximation. Sofic…
The translation of written language has been known since the 3rd century BC; however, its necessity has become increasingly common in the information age. Today, many translators exist, based on encoder-decoder deep architectures,…
Zipf's law states that sequential frequencies of words in a text correspond to a power function. Its probabilistic model is an infinite urn scheme with asymptotically power distribution. The exponent of this distribution must be estimated.…
We consider the situation when a globally defined four-dimensional field system is separated on two entangled sub-systems by a dynamical (random) two-dimensional surface. The reduced density matrix averaged over ensemble of random surfaces…
We propose a generalisation of Gibbs' statistical mechanics into the domain of non-negligible phase space correlations. Derived are the probability distribution and entropy as a generalised ensemble average, replacing…
We propose a stochastic model for the number of different words in a given database which incorporates the dependence on the database size and historical changes. The main feature of our model is the existence of two different classes of…
Contextual entropy is a psycholinguistic measure capturing the anticipated difficulty of processing a word just before it is encountered. Recent studies have tested for entropy-related effects as a potential complement to well-known effects…
A general upper bound for topological entropy of switched nonlinear systems is constructed, using an asymptotic average of upper limits of the matrix measures of Jacobian matrices of strongly persistent individual modes, weighted by their…
Topological entropy is a common measure of the rate of mixing in a flow. It can be computed by partition methods, or by estimating the growth rate of material lines or other material elements. This requires detailed knowledge of the…
An efficient approach to the calculation of the $\epsilon$-entropy is proposed. The method is based on the idea of looking at the information content of a string of data, by analyzing the signal only at the instants when the fluctuations…
We review recent progress in understanding the meaning of mutual information in natural language. Let us define words in a text as strings that occur sufficiently often. In a few previous papers, we have shown that a power-law distribution…
We consider infinite sequences of superstable orbits (cascades) generated by systematic substitutions of letters in the symbolic dynamics of one-dimensional nonlinear systems in the logistic map universality class. We identify the…
We construct the generalized entropy optimized by a given arbitrary statistical distribution with a finite linear expectation value of a random quantity of interest. This offers, via the maximum entropy principle, a unified basis for a…
We prove that all ($\alpha$-$\beta$)-shifts with $0\le \alpha<1$ and $\beta>2$ are saturated, that is, for any invariant measure, the topological entropy of the set of generic points coincides with the metric entropy.
The average uncertainty associated with words is an information-theoretic concept at the heart of quantitative and computational linguistics. The entropy has been established as a measure of this average uncertainty - also called average…
Entropy, its production, and its change in a dynamical system can be understood from either a fully stochastic dynamic description or from a deterministic dynamics exhibiting chaotic behavior. By taking the former approach based on the…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…