Related papers: Radical scaling: beyond our feet and fingers

Scaling laws, a defining feature of deep learning, reveal a striking power-law improvement in model performance with increasing dataset and model size. Yet, their mathematical origins, especially the scaling exponent, have remained elusive.…

By the early 1960's advances in statistical physics had established the existence of universality classes for systems with second-order phase transitions and characterized these by critical exponents which are different to the classical…

Scaling laws illuminate Nature's fundamental biological principles and guide bioinspired materials and structural designs. In simple cases they are based on the fundamental principle that all laws of nature remain unchanged (i.e.,…

Scale invariance is a central organizing principle in physics, underlying phenomena that range from critical behaviour in statistical mechanics to transport and chaos in nonlinear dynamical systems. Here we present a unified and physically…

Extreme events can come either from point processes, when the size or energy of the events is above a certain threshold, or from time series, when the intensity of a signal surpasses a threshold value. We are particularly concerned by the…

In the 1960's, four famous scaling relations were developed which relate the six standard critical exponents describing continuous phase transitions in the thermodynamic limit of statistical physics models. They are well understood at a…

The size of large cliff failures may be described in several ways, for instance considering the horizontal eroded area at the cliff top and the maximum local retreat of the coastline. Field studies suggest that, for large failures, the…

Cities are often compared through scaling laws, usually expressed as power-law relations between population size and aggregate urban quantities related to infrastructure, socioeconomic activity, or environmental impacts. These laws are…

Neural scaling laws define a predictable relationship between a model's parameter count and its performance after training in the form of a power law. However, most research to date has not explicitly investigated whether scaling laws can…

Analyses of urban scaling laws assume that observations in different cities are independent of the existence of nearby cities. Here we introduce generative models and data-analysis methods that overcome this limitation by modelling…

The population loss of trained deep neural networks often follows precise power-law scaling relations with either the size of the training dataset or the number of parameters in the network. We propose a theory that explains the origins of…

An analytic perturbation theory is suggested in order to find finite-size corrections to the scaling power laws. In the frame of this theory it is shown that the first order finite-size correction to the scaling power laws has following…

Interacting physical systems in the neighborhood of criticality (and massive continuum field theories) can often be characterized by just two physical scales: a (macroscopic) correlation length and a (microscopic) interaction range, related…

Deep neural networks exhibit empirical neural scaling laws, with error decreasing as a power law with increasing model or data size, across a wide variety of architectures, tasks, and datasets. This universality suggests that scaling laws…

Neural scaling laws describe how the performance of deep neural networks scales with key factors such as training data size, model complexity, and training time, often following power-law behaviors over multiple orders of magnitude. Despite…

When training deep neural networks, a model's generalization error is often observed to follow a power scaling law dependent both on the model size and the data size. Perhaps the best known example of such scaling laws are for…

We demonstrate the emergence of scaling laws in the benchmark top versus QCD jet classification problem in collider physics. Six distinct physically-motivated classifiers exhibit power-law scaling of the binary cross-entropy test loss as a…

The objective of statistical physics is to understand macroscopic behavior of a many-body system from the interactions of the constituents of that system. When many-body systems reach critical states, simple universal and scaling behaviors…

Scaling laws for characteristic length scales (in time or in the model parameters) are both experimentally robust and accessible for rigorous analysis. In multiscale situations cross--overs between different scaling laws are observed. We…

Scaling laws describe how learning performance improves with data, compute, or training time, and have become a central theme in modern deep learning. We study this phenomenon in a canonical nonlinear model: phase retrieval with anisotropic…