Related papers: Collective evolution of weights in wide neural net…

We study hyperbolic systems of one-dimensional partial differential equations under general, possibly non-local boundary conditions. A large class of evolution equations, either on individual 1-dimensional intervals or on general networks,…

We study degenerate fully nonlinear free transmission problems, where the degeneracy rate varies in the domain. We prove optimal pointwise regularity depending on the degeneracy rate. Our arguments consist of perturbation methods, relating…

A closed set of coupled equations of motion for the description of time-dependent electron transport is derived. It provides the time evolution of energy-resolved quantities constructed from non-equilibrium Green functions. By means of an…

Nonlinear, multiplicative Langevin equations for a complete set of slow variables in equilibrium systems are generally derived on the basis of the separation of time scales. The form of the equations is universal and equivalent to that…

Interpreting the learning dynamics of neural networks can provide useful insights into how networks learn and the development of better training and design approaches. We present an approach to interpret learning in neural networks by…

Gradient dynamics play a central role in determining the stability and generalization of deep neural networks. In this work, we provide an empirical analysis of how variance and standard deviation of gradients evolve during training,…

Linear layers in neural networks (NNs) trained by gradient descent can be expressed as a key-value memory system which stores all training datapoints and the initial weights, and produces outputs using unnormalised dot attention over the…

We prove the well-posedness of entropy solutions for a wide class of nonlocal transport equations with nonlinear mobility in one spatial dimension. The solution is obtained as the limit of approximations constructed via a deterministic…

We present a generic threshold model for the co-evolution of the structure of a network and the state of its nodes. We focus on regular directed networks and derive equations for the evolution of the system toward its absorbing state. It is…

Diffusion and flow-based generative models have achieved remarkable success in domains such as image synthesis, video generation, and natural language modeling. In this work, we extend these advances to weight space learning by leveraging…

In this paper, we study the linear transport model by adopting the deep learning method, in particular the deep neural network (DNN) approach. While the interest of using DNN to study partial differential equations is arising, here we adapt…

We develop a simple theoretical framework for the evolution of weighted networks that is consistent with a number of stylized features of real-world data. In our framework, the Barabasi-Albert model of network evolution is extended by…

We propose a new approach to the study of (nonlinear) growth and instability for semilinear evolution equations with compact nonlinearities. We show, in particular, that compact nonlinear perturbations of a linear evolution equation can be…

Real-world networks process structured connections since they have non-trivial vertex degree correlation and clustering. Here we propose a toy model of structure formation in real-world weighted network. In our model, a network evolves by…

Recent work on mode connectivity in the loss landscape of deep neural networks has demonstrated that the locus of (sub-)optimal weight vectors lies on continuous paths. In this work, we train a neural network that serves as a hypernetwork,…

In this paper we study the discrete coagulation--fragmentation models with growth, decay and sedimentation. We demonstrate the existence and uniqueness of classical global solutions provided the linear processes are sufficiently strong.…

In deep multi-task learning, weights of task-specific networks are shared between tasks to improve performance on each single one. Since the question, which weights to share between layers, is difficult to answer, human-designed…

We develop a theory of optimal transport for stationary random measures with a focus on stationary point processes and construct a family of distances on the set of stationary random measures. These induce a natural notion of interpolation…

We find a near detailed balance solution to the relativistic Boltzmann equation under the relaxation time approximation with a collision term which differs from the Anderson-Witting model and is dependent on the stationary observer. Using…

A new delay equation is introduced to describe the punctuated evolution of complex nonlinear systems. A detailed analytical and numerical investigation provides the classification of all possible types of solutions for the dynamics of a…