Related papers: Getting in the Zone for Successful Scalability
Data scaling has revolutionized fields like natural language processing and computer vision, providing models with remarkable generalization capabilities. In this paper, we investigate whether similar data scaling laws exist in robotics,…
When traversing a symmetry breaking second order phase transition at a finite rate, topological defects form whose number dependence on the quench rate is given by simple power laws. We propose a general approach for the derivation of such…
Finite-size scaling is a key tool in statistical physics, used to infer critical behavior in finite systems. Here we use the analogous concept of finite-time scaling to describe the bifurcation diagram at finite times in discrete dynamical…
The problem of learning parallel computer performance is investigated in the context of multicore processors. Given a fixed workload, the effect of varying system configuration on performance is sought. Conventionally, the performance…
Why do larger language models generalize better? To investigate this question, we develop generalization bounds on the pretraining objective of large language models (LLMs) in the compute-optimal regime, as described by the Chinchilla…
We study systems with a continuous phase transition that tune their parameters to maximize a quantity that diverges solely at a unique critical point. Varying the size of these systems with dynamically adjusting parameters, the same…
We generalize Huberman-Rudnick universal scaling law for all periodic windows of the logistic map and show the robustness of $q$-Gaussian probability distributions in the vicinity of chaos threshold. Our scaling relation is universal for…
Scaling laws, a defining feature of deep learning, reveal a striking power-law improvement in model performance with increasing dataset and model size. Yet, their mathematical origins, especially the scaling exponent, have remained elusive.…
Scaling laws describe how language models improve with additional data, parameters, and compute. While widely used, they are typically measured on aggregate test sets. Aggregate evaluations yield clean trends but average over heterogeneous…
Scale invariance is a central organizing principle in physics, underlying phenomena that range from critical behaviour in statistical mechanics to transport and chaos in nonlinear dynamical systems. Here we present a unified and physically…
We show that steady-state expectation values predicted by the universal Lindblad equation (ULE) are accurate up to bounded corrections that scale linearly with the effective system-bath coupling, $\Gamma$ (second order in the microscopic…
Scaling laws are used to plan multi-million-dollar training runs, but fitting those laws can itself cost millions. In modern large-scale workflows, assembling a sufficiently informative set of pilot experiments is already a major…
Downstream scaling laws aim to predict task performance at larger scales from the model's performance at smaller scales. Whether such prediction should be possible is unclear: some works discover clear linear scaling trends after simple…
Universality is key to the theory of phase transition stating that the equilibrium properties of observables near a phase transition can be classified according to few critical exponents. These exponents rule an universal scaling behaviour…
The existence of universal scaling in the vicinity of the jamming transition of sheared granular materials is predicted by a phenomenology. The critical exponents are explicitly determined, which are independent of the spatial dimension.…
Certain frustrated systems, including spin ice and dimer models, exhibit a Coulomb phase at low temperatures, with power-law correlations and fractionalized monopole excitations. Transitions out of this phase, at which the effective gauge…
With the establishment of cloud computing as the environment of choice for most modern applications, auto-scaling is an economic matter of great importance. For applications like stream computing that process ever changing amounts of data,…
Dynamic ride-sharing services, including ride-pooling offered by ride-hailing platforms and demand-responsive buses, have become an essential part of urban mobility systems. These services cater to personalized and on-demand mobility…
Ultra-large scale (ULS) systems are becoming pervasive. They are inherently complex, which makes their design and control a challenge for traditional methods. Here we propose the design and analysis of ULS systems using measures of…
The concept of universality has shaped our understanding of many-body physics, but is mostly limited to homogenous systems. Here, we present a study of universality on a non-homogeneous graph, the long-range diluted graph (LRDG). Its…