Related papers: How Long Does Infinite Width Last? Signal Propagat…

Neural Tangent Kernel (NTK) theory is widely used to study the dynamics of infinitely-wide deep neural networks (DNNs) under gradient descent. But do the results for infinitely-wide networks give us hints about the behavior of real…

Capacity scaling laws are analyzed in an underwater acoustic network with $n$ regularly located nodes on a square, in which both bandwidth and received signal power can be limited significantly. A narrow-band model is assumed where the…

Despite the widespread practical success of deep learning methods, our theoretical understanding of the dynamics of learning in deep neural networks remains quite sparse. We attempt to bridge the gap between the theory and practice of deep…

Neural Tangent Kernel (NTK) is widely used to analyze overparametrized neural networks due to the famous result by Jacot et al. (2018): in the infinite-width limit, the NTK is deterministic and constant during training. However, this result…

The dynamical evolution of the surface height is controlled by either a linear or a nonlinear Langevin equation, depending on the underlying microscopic dynamics, and is often done theoretically using stochastic coarse-grained growth…

We consider fully connected feed-forward deep neural networks (NNs) where weights and biases are independent and identically distributed as symmetric centered stable distributions. Then, we show that the infinite wide limit of the NN, under…

Recurrent neural networks are capable of learning the dynamics of an unknown nonlinear system purely from input-output measurements. However, the resulting models do not provide any stability guarantees on the input-output mapping. In this…

In this work, we investigate an intriguing and prevalent phenomenon of diffusion models which we term as "consistent model reproducibility": given the same starting noise input and a deterministic sampler, different diffusion models often…

A deep equilibrium model uses implicit layers, which are implicitly defined through an equilibrium point of an infinite sequence of computation. It avoids any explicit computation of the infinite sequence by finding an equilibrium point…

A problem of finding the linear theory satisfaction limits in propagation of the internal gravity waves is considered. It is evident that internal gravity waves excitation, propagation in actual practice is highly nonlinear phenomenon.…

A recurrent neural network (RNN) is a widely used deep-learning network for dealing with sequential data. Imitating a dynamical system, an infinite-width RNN can approximate any open dynamical system in a compact domain. In general, deep…

This paper investigates the approximation power of three types of random neural networks: (a) infinite width networks, with weights following an arbitrary distribution; (b) finite width networks obtained by subsampling the preceding…

Approximating nonlinear dynamics with a truncated perturbative expan- sion may be accurate for a while, but it in general breaks down at a long time scale that is one over the small expansion parameter. There are interesting occasions in…

The mechanisms of information transmission are investigated in a lattice of coupled continuous maps, by analyzing the propagation of both finite and infinitesimal disturbances. Two distinct regimes are detected: in the former case, both…

Two of the most popular parallel-in-time methods are Parareal and multigrid-reduction-in-time (MGRIT). Recently, a general convergence theory was developed in Southworth (2019) for linear two-level MGRIT/Parareal that provides necessary and…

The expressive power of neural networks is important for understanding deep learning. Most existing works consider this problem from the view of the depth of a network. In this paper, we study how width affects the expressiveness of neural…

Deep neural networks have achieved remarkable success in practice, yet a mechanistic understanding of how features evolve during training remains incomplete, especially in the large-depth limit. For ResNets under depth-$\mu$P scaling, prior…

Unstable homoepitaxy on rough substrates is treated within a linear continuum theory. The time dependence of the surface width $W(t)$ is governed by three length scales: The characteristic scale $l_0$ of the substrate roughness, the terrace…

For almost 70 years, researchers have typically selected the width of neural networks' layers either manually or through automated hyperparameter tuning methods such as grid search and, more recently, neural architecture search. This paper…

Flat band systems can yield interesting phenomena, such as dispersion suppression of waves with frequency at the band. While linear transport vanishes, the corresponding nonlinear case is still an open question. Here, we study power…