Related papers: Scaling laws for nonlinear dynamical models of art…

In this paper a nonlinear coupled Schrodinger system in the presence of mixed cubic and superlinear power laws is considered. A non standard numerical method is developed to approximate the solutions in higher dimensional case. The idea…

Computed-torque control requires a very precise dynamical model of the robot for compensating the manipulator dynamics. This allows reduction of the controller's feedback gains resulting in disturbance attenuation and other advantages.…

Physical modelling synthesis aims to generate audio from physical simulations of vibrating structures. Thin elastic plates are a common model for drum membranes. Traditional numerical methods like finite differences and finite elements…

As neural networks continue to grow in size but datasets might not, it is vital to understand how much performance improvement can be expected: is it more important to scale network size or data volume? Thus, neural network scaling laws,…

A central challenge in neuroscience is understanding how neural system implements computation through its dynamics. We propose a nonlinear time series model aimed at characterizing interpretable dynamics from neural trajectories. Our model…

Predicting material properties is crucial for designing better batteries, semiconductors, and medical devices. Deep learning helps scientists quickly find promising materials by predicting their energy, forces, and stresses. Companies scale…

We propose a novel scaling law for general-purpose decoder-only language models (LMs) trained on multilingual data, tackling the problem of balancing languages during multilingual pretraining. A primary challenge in studying multilingual…

In many longitudinal settings, economic theory does not guide practitioners on the type of restrictions that must be imposed to solve the rotational indeterminacy of factor-augmented linear models. We study this problem and offer several…

Unsteady aerodynamic models are necessary to accurately simulate forces and develop feedback controllers for wings in agile motion; however, these models are often high dimensional or incompatible with modern control techniques. Recently,…

We study synchronization of nonlinear systems that satisfy an incremental passivity property. We consider the case where the control input is subject to a class of disturbances, including constant and sinusoidal disturbances with unknown…

We consider the solvable neural scaling model with three parameters: data complexity, target complexity, and model-parameter-count. We use this neural scaling model to derive new predictions about the compute-limited, infinite-data scaling…

In machine learning, the scaling law describes how the model performance improves with the model and data size scaling up. From a learning theory perspective, this class of results establishes upper and lower generalization bounds for a…

We introduce compression laws for language language models (LLMs). While recent scaling laws have sought to understand how LLMs scale with respect to model size, pre-training data, and computational resources, we focus on understanding how…

Scaling law principles indicate a power-law correlation between loss and variables such as model size, dataset size, and computational resources utilized during training. These principles play a vital role in optimizing various aspects of…

Scaling laws, a defining feature of deep learning, reveal a striking power-law improvement in model performance with increasing dataset and model size. Yet, their mathematical origins, especially the scaling exponent, have remained elusive.…

We demonstrate the emergence of scaling laws in the benchmark top versus QCD jet classification problem in collider physics. Six distinct physically-motivated classifiers exhibit power-law scaling of the binary cross-entropy test loss as a…

Stylized models of the neurodynamics that underpin sensory motor control in animals are proposed and studied. The voluntary motions of animals are typically initiated by high level intentions created in the primary cortex through a…

At a continuous transition into a nonunique absorbing state, particle systems may exhibit nonuniversal critical behavior, in apparent violation of hyperscaling. We propose a generalized scaling theory for dynamic critical behavior at a…

Scaling laws guide the development of large language models (LLMs) by offering estimates for the optimal balance of model size, tokens, and compute. More recently, loss-to-loss scaling laws that relate losses across pretraining datasets and…

The vast majority of systems of practical interest are characterised by nonlinear dynamics. This renders the control and optimization of such systems a complex task due to their nonlinear behaviour. Additionally, standard methods such as…