Related papers: Anomalous Fluctuations in Autoregressive Models wi…
Students have a limited time to study and are typically ineffective at allocating study time. Machine-directed study strategies that identify which items need reinforcement and dictate the spacing of repetition have been shown to help…
Self-similar dynamical processes are characterized by a growing length scale $\xi$ which increases with time as $\xi \sim t^{1/z}$, where z is the dynamical exponent. The best known example is a simple random walk with z=2. Usually such…
Autoregressive Transformers are strong language models but incur O(T) complexity during per-token generation due to the self-attention mechanism. Recent work proposes kernel-based methods to approximate causal self-attention by replacing it…
Recent empirical studies have demonstrated long-memory in the signs of orders to buy or sell in financial markets [2, 19]. We show how this can be caused by delays in market clearing. Under the common practice of order splitting, large…
Let M be a compact Riemannian manifold with smooth boundary. We obtain the exact long time asymptotic behaviour of the heat kernel on abelian coverings of M with mixed Dirichlet and Neumann boundary conditions. As an application, we study…
The role of memory time fluctuations for instabilities in random media is considered. It is shown that fluctuations can result in infinitely fast growth of statistical moments. The effect is demonstrated in the framework of light…
We study the time correlation function of a density field in two-dimensional driven diffusive systems within the framework of fluctuating hydrodynamics. It is found that the time correlation exhibits power-law behavior in an intermediate…
Consider first a memoryless population model described by the usual branching process with a given mean reproduction matrix on a finite space of types. Motivated by the consequences of atavism in Evolutionary Biology, we are interested in a…
There is a recent trend in machine learning to increase model quality by growing models to sizes previously thought to be unreasonable. Recent work has shown that autoregressive generative models with cross-entropy objective functions…
We propose a simple mechanism by which scaling laws emerge from feature learning in multi-layer networks. We study a high-dimensional hierarchical target that is a globally high-degree function, but that can be represented by a combination…
Many natural phenomena can be described by power-laws. A closer look at various experimental data reveals more or less significant deviations from a 1/f spectrum. We exemplify such cases with phenomena offered by molecular biology, cell…
Based on the Langevin description of the Continuous Time Random Walk (CTRW), we consider a generalization of CTRW in which the waiting times between the subsequent jumps are correlated. We discuss the cases of exponential and slowly…
We study by theoretical analysis and by direct numerical simulation the dynamics of a wide class of asynchronous stochastic systems composed of many autocatalytic degrees of freedom. We describe the generic emergence of truncated power laws…
We propose a mechanism to produce fluctuations in the viscosity parameter ($\alpha$) in differetially rotating discs. We carried out a nonlinear analysis of a general accretion flow, where any perturbation on the background $\alpha$ was…
We characterize the power-law asymptotics of learning curves for Gaussian process regression (GPR) under the assumption that the eigenspectrum of the prior and the eigenexpansion coefficients of the target function follow a power law. Under…
Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in…
Training large models is both resource-intensive and time-consuming, making it crucial to understand the quantitative relationship between model performance and hyperparameters. In this paper, we present an empirical law that describes how…
We consider a dynamical system which has a stable attractor and which is perturbed by an additive noise. Under some quite typical conditions, the fluctuations from the attractor are intermittent and have a probability distribution with…
We study the motion of a one-dimensional particle which reverses its direction of acceleration stochastically. We focus on two contrasting scenarios, where the waiting-times between two consecutive acceleration reversals are drawn from (i)…
An analytical expression is derived for the transition path time distribution for a one-dimensional particle crossing of a parabolic barrier. Two cases are analyzed: (i) A non-Markovian process described by a generalized Langevin equation…