Related papers: Collective evolution of weights in wide neural net…
We investigate an operator renormalization group method to extract and describe the relevant degrees of freedom in the evolution of partial differential equations. The proposed renormalization group approach is formulated as an analytical…
We consider the optimization of a dynamical system by switching at discrete time points between abstract evolution equations composed by nonlinearly perturbed strongly continuous semigroups, nonlinear state reset maps at mode transition…
We study the problem of transfer learning and fine-tuning in linear models for both regression and binary classification. In particular, we consider the use of stochastic gradient descent (SGD) on a linear model initialized with pretrained…
Transport is an important function in many network systems and understanding its behavior on biological, social, and technological networks is crucial for a wide range of applications. However, it is a property that is not well-understood…
We consider a distributed estimation method in a setting with heterogeneous streams of correlated data distributed across nodes in a network. In the considered approach, linear models are estimated locally (i.e., with only local data)…
The rate equations are used to study the scale-free behavior of the weight distribution in evolving networks whose topology is determined only by degrees of preexisting vertices. An analysis of these equations shows that the degree…
Using rough path techniques, we provide a priori estimates for the output of Deep Residual Neural Networks in terms of both the input data and the (trained) network weights. As trained network weights are typically very rough when seen as…
Nonlinear response occurs naturally when a strong perturbation takes a system far from equilibrium. Despite of its omnipresence in nanoscale systems, it is difficult to predict in a general and efficient way. Here we introduce a way to…
This paper aims to investigate a multi-dimensional transport equation with nonlocal velocity and fractional dissipation.The balance between the nonlinearity and dissipation gives rise to three different cases, namely the subcritical,…
Understanding how neural networks learn remains one of the central challenges in machine learning research. From random at the start of training, the weights of a neural network evolve in such a way as to be able to perform a variety of…
We complete a full classification of non-degenerate traveling waves of scalar balance laws from the point of view of spectral and nonlinear stability/instability under (piecewise) smooth perturbations. A striking feature of our analysis is…
We propose a new evolutionary dynamics for population games with a discrete strategy set, inspired by the theory of optimal transport and Mean field games. The dynamics can be described as a Fokker-Planck equation on a discrete strategy…
We prove compactness and hence existence for solutions to a class of non linear transport equations. The corresponding models combine the features of linear transport equations and scalar conservation laws. We introduce a new method which…
The success of deep neural networks hinges on our ability to accurately and efficiently optimize high-dimensional, non-convex functions. In this paper, we empirically investigate the loss functions of state-of-the-art networks, and how…
We generalize Lyons' rough paths theory in order to give a pathwise meaning to some nonlinear infinite-dimensional evolution equation associated to an analytic semigroup and driven by an irregular noise. As an illustration, we discuss a…
We derive the system of differential equations for the gradient flow characterizing the training process of linear in-context learning in full generality. Next, we explore the geometric structure of the gradient flows in two instances,…
Highly-optimized complex transport networks serve crucial functions in many man-made and natural systems such as power grids and plant or animal vasculature. Often, the relevant optimization functional is non-convex and characterized by…
We investigate the non-equilibrium properties of an N-component scalar field theory. The time evolution of the correlation functions for an arbitrary ensemble of initial conditions is described by an exact functional differential equation.…
Logistic equations play a pivotal role in the study of any non linear evolution process exhibiting growth and saturation. The interest for the phenomenology, they rule, goes well beyond physical processes and cover many aspects of ecology,…
This paper considers traces at the initial time for solutions of evolution equations with local or non-local derivatives in vector-valued $L_p$ spaces with $A_p$ weight. To achieve this, we begin by introducing a generalized real…