Related papers: Understanding scaling through history-dependent pr…
A steadily increasing body of evidence suggests that the brain performs probabilistic inference to interpret and respond to sensory input and that trial-to-trial variability in neural activity plays an important role. The neural sampling…
The frequency distributions of DNA k-mers are shaped by fundamental biological processes and offer a window into genome structure and evolution. Inspired by analogies to natural language, prior studies have attempted to model genomic k-mer…
Most data processing techniques, applied to biomedical and sociological time series, are only valid for random fluctuations that are stationary in time. Unfortunately, these data are often non stationary and the use of techniques of…
Regionalization aims to partition a spatial domain into contiguous regions that share similar characteristics, enabling more effective spatial analysis, policy making, and resource management. Existing approaches for spatial regionalization…
We extract the individual names of persons mentioned in news reports from a Philippine-based daily in the English language from 2010-2012. Names are extracted using a learning algorithm that filters adjacent capitalized words and runs it…
Multiscale phenomena that evolve on multiple distinct timescales are prevalent throughout the sciences. It is often the case that the governing equations of the persistent and approximately periodic fast scales are prescribed, while the…
The probabilistic Latent Semantic Indexing model assumes that the expectation of the corpus matrix is low-rank and can be written as the product of a topic-word matrix and a word-document matrix. In this paper, we study the estimation of…
The problem of compression in standard information theory consists of assigning codes as short as possible to numbers. Here we consider the problem of optimal coding -- under an arbitrary coding scheme -- and show that it predicts Zipf's…
The spectrum of the generator (Kolmogorov operator) of a diffusion process, referred to as the Ruelle-Pollicott (RP) spectrum, provides a detailed characterization of correlation functions and power spectra of stochastic systems via…
We consider an infinite-dimensional stochastic clustering model on $\mathbb{R}$. In discrete time, each point of a unit-intensity simple point process moves halfway toward either of its left or right neighbors, chosen uniformly at random.…
We introduce a stochastic model to explain a double power-law distribution which exhibits two different Paretian behaviors in the upper and the lower tail and widely exists in social and economic systems. The model incorporates fitness…
A new model of search based on stochastic resetting is introduced, wherein rate of resets depends explicitly on time elapsed since the beginning of the process. It is shown that rate inversely proportional to time leads to paradoxical…
Many stochastic complex systems are characterized by the fact that their configuration space doesn't grow exponentially as a function of the degrees of freedom. The use of scaling expansions is a natural way to measure the asymptotic growth…
We introduce a non-growth model that generates the power-law distribution with the Zipf exponent. There are N elements, each of which is characterized by a quantity, and at each time step these quantities are redistributed through binary…
Zipf's law predicts a power-law relationship between word rank and frequency in language communication systems and has been widely reported in a variety of natural language processing applications. However, the emergence of natural language…
Scale-free dynamics in physical and biological systems can arise from a variety of causes. Here, we explore a branching process which leads to such dynamics. We find conditions for the appearance of power laws and study quantitatively what…
Diffusion models generate high-quality synthetic data. They operate by defining a continuous-time forward process which gradually adds Gaussian noise to data until fully corrupted. The corresponding reverse process progressively "denoises"…
Despite the recent successes of deep learning in natural language processing (NLP), there remains widespread usage of and demand for techniques that do not rely on machine learning. The advantage of these techniques is their…
The concept of memory is of central importance for characterizing complex systems and phenomena. Presence of long-term memories indicates how their dynamics can be less sensitive to initial conditions compared to the chaotic cases. On the…
Past work has established scaling laws that predict the performance of a neural language model (LM) as a function of its parameter count and the number of tokens it's trained on, enabling optimal allocation of a fixed compute budget. Are…