Related papers: Getting in the Zone for Successful Scalability

We expand on the investigation of the universal scaling properties in the early time behaviour of fast but smooth quantum quenches in a general $d$-dimensional conformal field theory deformed by a relevant operator of dimension $\Delta$…

Universal scaling laws govern the density of topological defects generated while crossing an equilibrium continuous phase transition. The Kibble-Zurek mechanism (KZM) predicts the dependence on the quench time for slow quenches. By…

The performance of a language model has been shown to be effectively modeled as a power-law in its parameter count. Here we study the scaling behaviors of Routing Networks: architectures that conditionally use only a subset of their…

In this work I present a generalization of Amdahl's law on the limits of a parallel implementation with many processors. In particular I establish some mathematical relations involving the number of processors and the dimension of the…

Neural scaling laws aim to characterize how out-of-sample error behaves as a function of model and training dataset size. Such scaling laws guide allocation of a computational resources between model and data processing to minimize error.…

Phenomenological scaling arguments suggest the existence of universal amplitudes in the finite-size scaling of certain correlation lengths in strongly anisotropic or dynamical phase transitions. For equilibrium systems, provided that…

In this paper, we systematically study the work statistics for quantum phase transition. For a quantum system approached by an anisotropic conformal field theory near the critical point, the driving protocols is divided into three different…

Networks of interconnected materials permeate throughout nature, biology, and technology due to exceptional mechanical performance. Despite the importance of failure resistance in network design and utility, no existing physical model…

The analysis of the radial distribution function of a system provides a possible procedure for uncovering interaction rules between individuals out of collective movement patterns. This approach from classical statistical mechanics has…

Neural scaling laws have garnered significant interest due to their ability to predict model performance as a function of increasing parameters, data, and compute. In this work, we propose a simple statistical ansatz based on memorization…

Scale has been a major driving force in improving machine learning performance, and understanding scaling laws is essential for strategic planning for a sustainable model quality performance growth, long-term resource planning and…

Effective LLM training depends on predictable scaling of key quantities -- such as final loss and optimal hyperparameters -- with model and dataset size. Qiu et al. (2025) recently showed that this predictability can extend beyond scalars:…

For many driven-nonequilibrium systems, the probability distribution functions of magnitude and recurrence-time of large events follow a powerlaw indicating a strong temporal correlation. In this paper we argue why these probability…

Using the geometric entanglement measure, we study the scaling of multipartite entanglement in several 1D models at criticality, specifically the linear harmonic chain and the XY spin chain encompassing both the Ising and XX critical…

We consider a noninteracting disordered system designed to model particle diffusion, relaxation in glasses, and impurity bands of semiconductors. Disorder originates in the random spatial distribution of sites. We find strong numerical…

Scaling laws predict the loss of a target machine learning model by extrapolating from easier-to-train models with fewer parameters or smaller training sets. This provides an efficient way for practitioners and researchers alike to compare…

Many different definitions of computational universality for various types of dynamical systems have flourished since Turing's work. We propose a general definition of universality that applies to arbitrary discrete time symbolic dynamical…

Dimension reduction is the process of embedding high-dimensional data into a lower dimensional space to facilitate its analysis. In the Euclidean setting, one fundamental technique for dimension reduction is to apply a random linear map to…

In this paper, we investigate what constitutes a good representation or model in self-supervised learning (SSL). We argue that a good representation should exhibit universality, characterized by three essential properties: discriminability,…

Critical states are sometimes identified experimentally through power-law statistics or universal scaling functions. We show here that such features naturally emerge from networks in self-sustained irregular regimes away from criticality.…