Related papers: Getting in the Zone for Successful Scalability
We consider the shape and topology optimization problem to design a structure that minimizes a weighted sum of material consumption and (linearly) elastic compliance under a fixed given boundary load. As is well-known, this problem is in…
Traffic congestion, a daily frustration for millions and a multi-billion dollar drain on economies, has long resisted deep physical understanding. While simple theoretical models of traffic flow have suggested connections to critical…
The dynamics of many-body systems spanning condensed matter, cosmology, and beyond is hypothesized to be universal when the systems cross continuous phase transitions. The universal dynamics is expected to satisfy a scaling symmetry of…
Neural scaling laws characterize how model performance improves as the model size scales up. Inspired by empirical observations, we introduce a resource model of neural scaling. A task is usually composite hence can be decomposed into many…
The nature of statistics, statistical mechanics and consequently the thermodynamics of stochastic systems is largely determined by how the number of states $W(N)$ depends on the size $N$ of the system. Here we propose a scaling expansion of…
Bifurcations are one of the most remarkable features of dynamical systems. Corral et al. [Sci. Rep. 8(11783), 2018] showed the existence of scaling laws describing the transient (finite-time) dynamics in discrete dynamical systems close to…
We use deposition models of kinetic roughening of a growing surface to introduce the concepts of universality and scaling and to analyze the qualitative and quantitative role of different parameters. In particular, we focus on two classes…
We consider the \mnk{classical} problem of a controller activating (or sampling) sequentially from a finite number of $N \geq 2$ populations, specified by unknown distributions. Over some time horizon, at each time $n = 1, 2, \ldots$, the…
The theory of spin models intersects with condensed matter physics, complex systems, graph theory, combinatorial optimization, computational complexity and neural networks. Many ensuing applications rely on the fact that complicated spin…
Linear temporal logic (LTL) is a specification language for finite sequences (called traces) widely used in program verification, motion planning in robotics, process mining, and many other areas. We consider the problem of learning LTL…
Scaling laws guide the development of large language models (LLMs) by offering estimates for the optimal balance of model size, tokens, and compute. More recently, loss-to-loss scaling laws that relate losses across pretraining datasets and…
We study the empirical scaling laws of a family of encoder-decoder autoregressive transformer models on the task of joint motion forecasting and planning in the autonomous driving domain. Using a 500 thousand hours driving dataset, we…
Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in…
We introduce a new concept called scalability to adaptive control in this paper. In particular, we analyze how to scale learning rates of adaptive weight update laws of various adaptive control schemes with respect to given command profiles…
A State Space Model (SSM) is a foundation model in time series analysis, which has recently been shown as an alternative to transformers in sequence modeling. In this paper, we theoretically study the generalization of SSMs and propose…
Slimmable networks are a family of neural networks that can instantly adjust the runtime width. The width can be chosen from a predefined widths set to adaptively optimize accuracy-efficiency trade-offs at runtime. In this work, we propose…
The growing need for affordable and accessible higher education is a major global challenge for the 21st century. Consequently, there is a need to develop a deeper understanding of the functionality and taxonomy of universities and colleges…
It is commonly believed that scaling language models should commit a significant space or time cost, by increasing the parameters (parameter scaling) or output tokens (inference-time scaling). We introduce the third and more…
The scaling properties of the roughness of surfaces grown by two different processes randomly alternating in time, are addressed. The duration of each application of the two primary processes is assumed to be independently drawn from given…
In mechanical engineering W\"ohler plots serve to measure the average number of load cycles before materials break, as a function of the maximal stress in each cycle. Although such plots are prevalent in engineering for more than 150 years,…