Related papers: Collective evolution of weights in wide neural net…
In this paper, we study the dynamics of gradient descent in learning neural networks for classification problems. Unlike in existing works, we consider the linearly non-separable case where the training data of different classes lie in…
We introduce the notion of globally updating evolution for a class of weighted networks, in which the weight of a link is characterized by the amount of data packet transport flowing through it. By noting that the packet transport over the…
This work examines the deep disconnect between existing theoretical analyses of gradient-based algorithms and the practice of training deep neural networks. Specifically, we provide numerical evidence that in large-scale neural network…
We present a general model for the growth of weighted networks in which the structural growth is coupled with the edges' weight dynamical evolution. The model is based on a simple weight-driven dynamics and a weights' reinforcement…
Complex network theory has been used to study complex systems. However, many real-life systems involve multiple kinds of objects . They can't be described by simple graphs. In order to provide complete information of these systems, we…
We consider a linear integro-differential equation which arises to describe both aggregation-fragmentation processes and cell division. We prove the existence of a solution $(\lb,\U,\phi)$ to the related eigenproblem. Such eigenelements are…
In this paper, we provide the first precise distributional characterization of gradient descent iterates for general multi-layer neural networks under the canonical single-index regression model, in the `finite-width proportional regime'…
Modeling stochastic dynamics from discrete observations is a key interdisciplinary challenge. Existing methods often fail to estimate the continuous evolution of probability densities from trajectories or face the curse of dimensionality.…
We investigate the spectral properties of linear-width feed-forward neural networks, where the sample size is asymptotically proportional to network width. Empirically, we show that the spectra of weight in this high dimensional regime are…
We prove that the evolution of weight vectors in online gradient descent can encode arbitrary polynomial-space computations, even in very simple learning settings. Our results imply that, under weak complexity-theoretic assumptions, it is…
We study evolution equations on networks that can be modeled by means of hyperbolic systems. We extend our previous findings in \cite{KraMugNic20} by discussing well-posedness under rather general transmission conditions that might be…
We propose a model for the growth of weighted networks that couples the establishment of new edges and vertices and the weights' dynamical evolution. The model is based on a simple weight-driven dynamics and generates networks exhibiting…
Inspired by scientific collaboration networks, especially our empirical analysis of the network of econophysicists, an evolutionary model for weighted networks is proposed. Both degree-driven and weight-driven models are considered.…
We provide a well-posedness theory for a class of nonlocal continuity equations on co-evolving graphs. We describe the connection among vertices through an edge weight function and we let it evolve in time, coupling its dynamics with the…
Many problems in machine learning involve calculating correspondences between sets of objects, such as point clouds or images. Discrete optimal transport provides a natural and successful approach to such tasks whenever the two sets of…
In this paper we consider the semi-discretization in space of a first order scalar transport equation. For the space discretization we use standard continuous finite elements. To obtain stability we add a penalty on the jump of the gradient…
From the smallest biological systems to the largest cosmological structures, spatial domains undergo expansion and contraction. Within these growing domains, diffusive transport is a common phenomenon. Mathematical models have been widely…
We study large deviations in the context of stochastic gradient descent for one-hidden-layer neural networks with quadratic loss. We derive a quenched large deviation principle, where we condition on an initial weight measure, and an…
We study the evolution of hidden-weight spectra in wide neural networks trained by (stochastic) gradient descent. We develop a two-level dynamical mean-field theory (DMFT) that jointly tracks bulk and outlier spectral dynamics for spiked…
Many existing transductive bounds rely on classical complexity measures that are computationally intractable and often misaligned with empirical behavior. In this work, we establish new representation-based generalization bounds in a…