Related papers: Maximizing entropy for power-free languages
A word is square-free if it does not contain nonempty factors of the form $XX$. In 1906 Thue proved that there exist arbitrarily long square-free words over a $3$-letter alphabet. It was proved recently [7] that among these words there are…
In this paper, we consider the problem of variable-length source coding allowing errors. The exponential moment of the codeword length is analyzed in the non-asymptotic regime and in the asymptotic regime. Our results show that the smooth…
Quantum many-body states that frequently appear in physics often obey an entropy scaling law, meaning that an entanglement entropy of a subsystem can be expressed as a sum of terms that scale linearly with its volume and area, plus a…
Thermodynamics can be formulated in either of two approaches, the phenomenological approach, which refers to the macroscopic properties of systems, and the statistical approach, which describes systems in terms of their microscopic…
Given a totally finite ordered alphabet $ A $, endowing the set of words over $ A $ with the alternating lexicographic order, we define a new class of Lyndon words. We study the fundamental properties of the associated symbolic dynamical…
Learning systems acquire structured internal representations from data, yet classical information-theoretic results state that deterministic transformations do not increase information. This raises a fundamental question: how can learning…
We study succinctness as a measure of the expressive power of transformers. Succinctness -- how compactly a formalism can describe a language relative to other formalisms -- is a classical notion in logic and automata theory. We prove that…
We give an upper bound for the topological entropy of maps on inverse limit spaces in terms of their set-valued components. In a special case of a diagonal map on the inverse limit space $\underleftarrow{\lim}(I,f)$, where every diagonal…
The notion of topological entropy can be conceptualized in terms of the number of forward trajectories that are distinguishable at resolution $\varepsilon$ within $T$ time units. It can then be formally defined as a limit of a limit…
Using the concept of discrete noiseless channels, it was shown by Shannon in A Mathematical Theory of Communication that the ultimate performance of an encoder for a constrained system is limited by the combinatorial capacity of the system…
In the context of state estimation under communication constraints, several notions of dynamical entropy play a fundamental role, among them: topological entropy and restoration entropy. In this paper, we present a theorem which…
We extend a generic class of systems which have previously been shown to spontaneously develop scaling (power law) distributions of their elementary degrees of freedom. While the previous systems were linear and exploded exponentially for…
We introduce a simple, yet novel entropy-based framework to drive token efficiency in large language models during reasoning tasks. Our approach uses Shannon entropy from token-level logprobs as a confidence signal to enable early stopping,…
We study the effective behavior of heterogeneous energies arising in the modeling of material voids in geometrically linear elastic materials. Specifically, we consider functionals featuring bulk terms depending on the symmetrized gradient…
The use of exponentials in linear logic greatly enhances its expressive power. In this paper we focus on nonassociative noncommutative multiplicative linear logic, and systematically explore modal axioms K, T, and 4 as well as the…
There are different ways of measuring diversity in complex systems. In particular, in language, lexical diversity is characterized in terms of the type-token ratio and the word entropy. We here investigate both diversity metrics in six…
The extension of thermodynamic principles to active matter remains a challenge due to the non-equilibrium nature inherent to active systems. In this study, we introduce a framework to assess entropy in our minimal macroscopic experiment…
We prove existence of maximal entropy measures for an open set of non-uniformly expanding local diffeomorphisms on a compact Riemannian manifold. In this context the topological entropy coincides with the logarithm of the degree, and these…
In this paper we study the ergodic theory of a robust non-uniformly expanding maps where no Markov assumption is required. We prove that the topological pressure is differentiable as a function of the dynamics and analytic with respect to…
We study the dynamical properties of ball expanding maps, a class of continuous self-maps defined on compact metric spaces. For a ball expanding map, we show that: (1) the set of periodic points is dense in the chain recurrent set; (2) if…