Related papers: Maximizing entropy for power-free languages
Natural language syntax yields an unbounded array of hierarchically structured expressions. We claim that these are used in the service of active inference in accord with the free-energy principle (FEP). While conceptual advances alongside…
We show that a certain type of quasi finite, conservative, ergodic, measure preserving transformation always has a maximal zero entropy factor, generated by predictable sets. We also construct a conservative, ergodic, measure preserving…
We study nonhyperbolic and transitive partially hyperbolic diffeomorphisms having a one-dimensional center. We prove joint flexibility with respect to entropy and center Lyapunov exponent for a broad class of these systems. Flexibility…
In this paper, we show that each expanding Thurston map $f : S^2\rightarrow S^2$ has $1+ deg f$ fixed points, counted with appropriate weight, where $ deg f$ denotes the topological degree of the map $f$. We then prove the equidistribution…
Maximum-entropy ensembles are key primitives in statistical mechanics from which thermodynamic properties can be derived. Over the decades, several approaches have been put forward in order to justify from minimal assumptions the use of…
We describe an ensemble of growing scale-free networks in an equilibrium framework, providing insight into why the exponent of empirical scale-free networks in nature is typically robust. In an analogy to thermostatistics, to describe the…
We prove that the topological entropy of any dominant rational self-map of a projective variety defined over a complete non-Archimedean field is bounded from above by the maximum of its dynamical degrees, thereby extending a theorem of…
We consider suspension flows with continuous roof function over the full shift $\Sigma$ on a finite alphabet. For any positive entropy subshift of finite type $Y \subset \Sigma$, we explictly construct a roof function such that the…
A convenient measure of a map or flow's chaotic action is the topological entropy. In many cases, the entropy has a homological origin: it is forced by the topology of the space. For example, in simple toral maps, the topological entropy is…
We construct a family of shift spaces with almost specification and multiple measures of maximal entropy. This answers a question from Climenhaga and Thompson [Israel J. Math. 192 (2012), no. 2, 785--817]. Elaborating on our examples we…
This paper proposes a novel entropy encoding technique for lossless data compression. Representing a message string by its lexicographic index in the permutations of its symbols results in a compressed version matching Shannon entropy of…
We prove that for every ergodic invariant measure with positive entropy of a continuous map on a compact metric space there is $\delta>0$ such that the dynamical $\delta$-balls have measure zero. We use this property to prove, for instance,…
We study the lexicographically least infinite $a/b$-power-free word on the alphabet of non-negative integers. Frequently this word is a fixed point of a uniform morphism, or closely related to one. For example, the lexicographically least…
Entropy of matter in a very strong gravity depends on cross-sectional area of the container of the system -- is being further bolstered by calculating entropy of a monoatomic gas kept under uniform strong gravity at Newtonian scale. This…
We show that for every linear toral automorphism, especially the non-hyperbolic ones, the entropies of ergodic measures form a dense set on the interval from zero to the topological entropy.
Zipf's law on word frequency is observed in English, French, Spanish, Italian, and so on, yet it does not hold for Chinese, Japanese or Korean characters. A model for writing process is proposed to explain the above difference, which takes…
Increasing the number $N$ of elements of a system typically makes the entropy to increase. The question arises on {\it what particular entropic form} we have in mind and {\it how it increases} with $N$. Thermodynamically speaking it makes…
We define some pointwise properties of topological dynamical systems and give pointwise conditions for such a system possesses positive topological entropy. We give sufficient conditions to obtain positive topological entropy for maps which…
We use large language models (LLMs) to uncover long-ranged structure in English texts from a variety of sources. The conditional entropy or code length in many cases continues to decrease with context length at least to $N\sim 10^4$…
Evaluating whether large language models (LLMs) capture the structure of natural language beyond local fluency remains an open challenge. Existing evaluation methods, largely based on task performance or short-context behavior, provide…