Related papers: Maximizing entropy for power-free languages
Active nematics are an important new paradigm in soft condensed matter systems. They consist of rod-like components with an internal driving force pushing them out of equilibrium. The resulting fluid motion exhibits chaotic advection, in…
Many complex generative systems use languages to create structured objects. We consider a model of random languages, defined by weighted context-free grammars. As the distribution of grammar weights broadens, a transition is found from a…
We study the optimization of ergodic averages for multi-valued dynamical systems, i.e. where points may have multiple different forward orbits. Under upper semi-continuity assumptions, we show that the maximum space average with respect to…
We consider partially hyperbolic diffeomorphisms $f$ with a one-dimensional central direction such that the unstable entropy exceeds the stable entropy. Our main result proves that such maps have a finite number of ergodic measures of…
We analyze the finite temperature behaviour of massless conformally coupled scalar fields in homogeneous lens spaces $S^3/{\mathbb Z}_p$. High and low temperature expansions are explicitly computed and the behavior of thermodynamic…
In this study, the output of large language models (LLM) is considered an information source generating an unlimited sequence of symbols drawn from a finite alphabet. Given the probabilistic nature of modern LLMs, we assume a probabilistic…
Motivated by the notion of intermediate dimensions introduced by Falconer et al., we introduce a continuum of topological entropies that are intermediate between the (Bowen) topological entropy and the lower and upper capacity topological…
Language prediction is constrained by informational entropy intrinsic to language, such that there exists a limit to how accurate any language model can become and equivalently a lower bound to language compression. The most efficient…
We prove that $\mathcal{C}^r$ maps with $r>1$ on a compact surface have symbolic extensions, i.e. topological extensions which are subshifts over a finite alphabet. More precisely we give a sharp upper bound on the so-called symbolic…
We characterize the words that can be mapped to arbitrarily high powers by injective morphisms. For all other words, we prove a linear upper bound for the highest power that they can be mapped to, and this bound is optimal up to a constant…
Maximum entropy models provide the least constrained probability distributions that reproduce statistical properties of experimental datasets. In this work we characterize the learning dynamics that maximizes the log-likelihood in the case…
Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…
The tensionless limit of classical string theory may be formulated as a topological theory on the world-sheet. A vector density carries geometrical information in place of an internal metric. It is found that path-integral quantization…
We consider the embedding problem in coding theory: given an independence (a code-related property) and an independent language $L$, find a maximal independent language containing $L$. We consider the case where the code-related property is…
We prove that systems satisfying the specification property are saturated in the sense that the topological entropy of the set of generic points of any invariant measure is equal to the measure-theoretic entropy of the measure. We study…
Let us consider a homogeneous Markov chain with discrete time and with a finite set of states $E_0,\ldots,E_n$ such that the state $E_0$ is absorbing, states $E_1,\ldots,E_n$ are nonrecurrent. The goal of this work is to study frequencies…
The topological entropy dimension is mainly used to distinguish the zero topological entropy systems. Two types of topological entropy dimensions, the classical entropy dimension and the Pesin entropy dimension, are investigated for…
The Maximum Entropy Modeling Toolkit supports parameter estimation and prediction for statistical language models in the maximum entropy framework. The maximum entropy framework provides a constructive method for obtaining the unique…
We show that a shift space on a finite alphabet with a non-uniform specification property can be modeled by a strongly positive recurrent countable-state Markov shift to which every equilibrium state lifts. In addition to uniqueness of the…
Let $ \beta $ be a real number less than -1. In this paper, we prove the uniqueness of the measure with maximal entropy of the negative $\beta$-shift. Endowed with the shift, this symbolic dynamical system is coded under certain conditions,…