Related papers: Intrinsic Complexity And Scaling Laws: From Random…

Many physical and mathematical models involve random fields in their input data. Examples are ordinary differential equations, partial differential equations and integro--differential equations with uncertainties in the coefficient…

We study universal traits which emerge both in real-world complex datasets, as well as in artificially generated ones. Our approach is to analogize data to a physical system and employ tools from statistical physics and Random Matrix Theory…

Learning arguably involves the discovery and memorization of abstract rules. The aim of this paper is to study associative memory mechanisms. Our model is based on high-dimensional matrices consisting of outer products of embeddings, which…

In many applications, random fields reflect uncertain parameters, and often their moments are part of the modeling process and thus well known. However, there are practical situations where this is simply not the case. Therefore, we do not…

Deep neural networks exhibit empirical neural scaling laws, with error decreasing as a power law with increasing model or data size, across a wide variety of architectures, tasks, and datasets. This universality suggests that scaling laws…

We consider random rectangles in $\mathbb{R}^2$ that are distributed according to a Poisson random measure, i.e., independently and uniformly scattered in the plane. The distributions of the length and the width of the rectangles are…

When training deep neural networks, a model's generalization error is often observed to follow a power scaling law dependent both on the model size and the data size. Perhaps the best known example of such scaling laws are for…

We present a new modeling paradigm for optimization that we call random field optimization. Random fields are a powerful modeling abstraction that aims to capture the behavior of random variables that live on infinite-dimensional spaces…

Neural scaling laws describe how the performance of deep neural networks scales with key factors such as training data size, model complexity, and training time, often following power-law behaviors over multiple orders of magnitude. Despite…

Many complex systems are characterized by intriguing spatio-temporal structures. Their mathematical description relies on the analysis of appropriate correlation functions. Functional integral techniques provide a unifying formalism that…

Power law generalized covariance functions provide a simple model for describing the local behavior of an isotropic random field. This work seeks to extend this class of covariance functions to spatial-temporal processes for which the…

The success of machine learning has resulted from its structured representation of data. Similar data have close internal representations as compressed codes for classification or emerged labels for clustering. We observe that the frequency…

Neural scaling laws establish a predictable relationship between model performance and data or compute, offering crucial guidance for resource allocation in new domains and tasks. Yet such laws are most needed precisely where they are…

We consider a broad class of Continuous Time Random Walks with large fluctuations effects in space and time distributions: a random walk with trapping, describing subdiffusion in disordered and glassy materials, and a L\'evy walk process,…

We compute spectra of sample auto-covariance matrices of second order stationary stochastic processes. We look at a limit in which both the matrix dimension $N$ and the sample size $M$ used to define empirical averages diverge, with their…

Intrinsic Gaussian Markov Random Fields (IGMRFs) can be used to induce conditional dependence in Bayesian hierarchical models. IGMRFs have both a precision matrix, which defines the neighbourhood structure of the model, and a precision, or…

The scaling properties of the roughness of surfaces grown by two different processes randomly alternating in time, are addressed. The duration of each application of the two primary processes is assumed to be independently drawn from given…

We consider a class of real numbers, a subset of irrational numbers and certain mathematical constants, for which the elements in the simple continued fraction appears to be random. As an illustrative example, one can consider $\pi = \{x_0,…

Real-world road networks have an approximate scale-invariance property; can one devise mathematical models of random networks whose distributions are {\em exactly} invariant under Euclidean scaling? This requires working in the continuum…

We begin with isotropic Gaussian random fields, and show how the Bochner-Godement theorem gives a natural way to describe their covariance structure. We continue with a study of Mat\'ern processes on Euclidean space, spheres, manifolds and…