Related papers: On the Entropy of Random Fibonacci Words
Recently Lewis Bowen introduced a notion of entropy for measure-preserving actions of a countable sofic group on a standard probability space admitting a generating partition with finite entropy. By applying an operator algebra perspective…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
On a compact group the Haar probability measure plays the role of uniform distribution. The entropy and rate distortion theory for this uniform distribution is studied. New results and simplified proofs on convergence of convolutions on…
We introduce a concept for random tilings which, comprising the conventional one, is also applicable to tiling ensembles without height representation. In particular, we focus on the random tiling entropy as a function of the tile…
We introduce and analyze a random tree model associated to Hoppe's urn. The tree is built successively by adding nodes to the existing tree when starting with the single root node. In each step a node is added to the tree as a child of an…
We present a simple structure based model of how words are formed from morphemes. The model explains two major empirical facts: the typical distribution of word lengths and the appearance of Zipf like rank frequency curves. In contrast to…
Today's probabilistic language generators fall short when it comes to producing coherent and fluent text despite the fact that the underlying models perform well under standard metrics, e.g., perplexity. This discrepancy has puzzled the…
We show that the $p$-adic valuation of the sequence of Fibonacci numbers is a $p$-regular sequence for every prime $p$. For $p \neq 2, 5$, we determine that the rank of this sequence is $\alpha(p) + 1$, where $\alpha(m)$ is the restricted…
Token-level reweighting is a simple yet effective mechanism for controlling supervised fine-tuning, but common indicators are largely one-dimensional: the ground-truth probability reflects downstream alignment, while token entropy reflects…
Observational entropy -- a quantity that unifies Boltzmann's entropy, Gibbs' entropy, von Neumann's macroscopic entropy, and the diagonal entropy -- has recently been argued to play a key role in a modern formulation of statistical…
The Tribonacci sequence $\mathbb{T}$ is the fixed point of the substitution $\sigma(a,b,c)=(ab,ac,a)$. In this note, we get the explicit expressions of all squares, and then establish the tree structure of the positions of repeated squares…
We show that essentially the Fibonacci sequence is the unique binary recurrence which contains infinitely many three-term arithmetic progressions. A criterion for general linear recurrences having infinitely many three-term arithmetic…
A mathematical interpretation of the usual definition of entropy (for a discrete probability distribution or a trace 1 positive operator) is given. This formulation makes some properties of entropy immediate.
Fix two words over the binary alphabet $\{0,1\}$, and generate iid Bernoulli$(p)$ bits until one of the words occurs in sequence. This setup, commonly known as Penney's ante, was popularized by Conway, who found (in unpublished work) a…
We find that the simple coupling of network growth to the position of a random walker on the network generates a traveling wave in the probability distribution of nodes visited by the walker. We argue that the entropy of this probability…
In this work we define a stochastic adding machine associated to the Fibonacci base and to a probabilities sequence $\overline{p}=(p_i)_{i\geq 1}$. We obtain a Markov chain whose states are the set of nonnegative integers. We study…
The focus of this paper is the random sequences in the form $\{X_{0},X_{1},$ $X_{n}=X_{n-2}+X_{n-1},n=2,3,..\dot{\}},$ referred to as Fibonacci Random Sequence (FRS). The initial random variables $X_{0}$ and $X_{1}$ are assumed to be…
The concept of entropy connects the number of possible configurations with the number of variables in large stochastic systems. Independent or weakly interacting variables render the number of configurations scale exponentially with the…
We explore connections between homotopy type theory and information theory through homotopy cardinality. We define probability types and random variable types, prove that homotopy cardinality respects dependent sums under truncation and…
We present two complementary proofs that, if the lengths of $n$ sticks are sampled at random, then the probability that no $p+1$ sticks can form a $(p+1)$-sided polygon can be expressed as the product of the reciprocals of a series of terms…