相关论文: A Paradoxical Property of the Monkey Book

Although information theoretic characterizations of human communication have become increasingly popular in linguistics, to date they have largely involved grafting probabilistic constructs onto older ideas about grammar. Similarities…

Power law is one of the the simplest forms of the relationship between different variables of a system. It leads naturally to the introduction of compound parameters describing physical properties of the system. Often one of the variables…

We study rank-frequency relations for phonemes, the minimal units that still relate to linguistic meaning. We show that these relations can be described by the Dirichlet distribution, a direct analogue of the ideal-gas model in statistical…

Evaluating whether large language models (LLMs) capture the structure of natural language beyond local fluency remains an open challenge. Existing evaluation methods, largely based on task performance or short-context behavior, provide…

In this article, it is shown specifically that natural system chance events as represented by theory predicted (a priori) probabilistic statements used in such realms as modern particle physics, among others, are only random relative to the…

Multiplicative random processes in (not necessaryly equilibrium or steady state) stochastic systems with many degrees of freedom lead to Boltzmann distributions when the dynamics is expressed in terms of the logarithm of the normalized…

We propose a stochastic model for the number of different words in a given database which incorporates the dependence on the database size and historical changes. The main feature of our model is the existence of two different classes of…

Consider a network of M >> 1 nodes connected by N >> 1 links, in which the distribution of the number of links per node follows a power law with exponent 0<\alpha <1. The power law is naturally truncated due to the fact that N is finite. A…

Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a COUNT associated with…

Language provides simple ways of communicating generalizable knowledge to each other (e.g., "Birds fly", "John hikes", "Fire makes smoke"). Though found in every language and emerging early in development, the language of generalization is…

Large Language Models (LLMs) have achieved unprecedented performance on many complex tasks, being able, for example, to answer questions on almost any topic. However, they struggle with other simple tasks, such as counting the occurrences…

It is known that the core of mathematics is natural numbers. And everything related to the natural number is interesting to mathematicians. In this paper, we draw parallels between natural numbers and elements of a non-numeric lexicographic…

We consider random walks on the set of all words over a finite alphabet such that in each step only the last two letters of the current word may be modified and only one letter may be adjoined or deleted. We assume that the transition…

Written language is complex. A written text can be considered an attempt to convey a meaningful message which ends up being constrained by language rules, context dependence and highly redundant in its use of resources. Despite all these…

A tanglegram consists of two binary rooted trees with the same number of leaves and a perfect matching between the leaves of the trees. We show that the two halves of a random tanglegram essentially look like two independently chosen random…

We begin with the extraordinary observation that the length distribution of 80 million proteins in UniProt, the Universal Protein Resource, measured in amino acids, is qualitatively identical to the length distribution of large collections…

We study the distributions of money in a simple closed economic system for different types of monetary transactions. We know that for arbitrary and random sharing but locally conserving money transactions, the money distribution goes to the…

Here we describe how some important scaling laws observed in the distribution of languages on Earth can emerge from a simple computer simulation. The proposed language dynamics includes processes of selective geographic colonization,…

In probability theory, there is a tendency to treat one random variable with a given distribution as being just as good as any other. By and large this is fine because probability is (mostly) concerned with distributional properties of…

We study probability distributions over free algebras of trees. Probability distributions can be seen as particular (formal power) tree series [Berstel et al 82, Esik et al 03], i.e. mappings from trees to a semiring K . A widely studied…