Related papers: Stochastic optimization on matrices and a graphon …
Spatially structured neural networks driven by jump diffusion noise with monotone coefficients, fully path dependent delay and with a disorder parameter are considered. Well-posedness for the associated McKean-Vlasov equation and a…
In a variety of problems originating in supervised, unsupervised, and reinforcement learning, the loss function is defined by an expectation over a collection of random variables, which might be part of a probabilistic model or the external…
As an enhanced version of existing results on Kac's propagation of chaos, which describes the convergence of mean-field particle systems to a system of independent McKean-Vlasov particles as the number of particles tends to infinity, we…
We study the optimal estimation of probability matrices of random graph models generated from graphons. This problem has been extensively studied in the case of step-graphons and H\"older smooth graphons. In this work, we characterize the…
We study the scaling limits of stochastic gradient descent (SGD) with constant step-size in the high-dimensional regime. We prove limit theorems for the trajectories of summary statistics (i.e., finite-dimensional functions) of SGD as the…
We consider in this work small random perturbations (of multiplicative noise type) of the gradient flow. We prove that under mild conditions, when the potential function is a Morse function with additional strong saddle condition, the…
In this paper, we characterize the noise of stochastic gradients and analyze the noise-induced dynamics during training deep neural networks by gradient-based optimizers. Specifically, we firstly show that the stochastic gradient noise…
A sequence of graphs with diverging number of nodes is a dense graph sequence if the number of edges grows approximately as for complete graphs. To each such sequence a function, called graphon, can be associated, which contains information…
Adaptive stochastic gradient algorithms in the Euclidean space have attracted much attention lately. Such explorations on Riemannian manifolds, on the other hand, are relatively new, limited, and challenging. This is because of the…
We prove limit theorems for systems of interacting diffusions on sparse graphs. For example, we deduce a hydrodynamic limit and the propagation of chaos property for the stochastic Kuramoto model with interactions determined by…
We consider the problem of minimizing a convex function that is evolving according to unknown and possibly stochastic dynamics, which may depend jointly on time and on the decision variable itself. Such problems abound in the machine…
We investigate a class of stochastic partial differential equations of reaction-diffusion type defined on graphs, which can be derived as the limit of SPDEs on narrow planar channels. In the first part, we demonstrate that this limit can be…
Large-scale networked systems typically operate under resource constraints, and it is also difficult to exactly obtain the network structure between nodes. To address these issues, this paper investigates a sparse optimal control for…
Pruning at Initialisation methods discover sparse, trainable subnetworks before training, but their theoretical mechanisms remain elusive. Existing analyses are often limited to finite-width statistics, lacking a rigorous characterisation…
This paper investigates asymptotic behaviors of gradient descent algorithms (particularly accelerated gradient descent and stochastic gradient descent) in the context of stochastic optimization arising in statistics and machine learning…
The emerging theory of graph limits exhibits an analytic perspective on graphs, showing that many important concepts and tools in graph theory and its applications can be described more naturally (and sometimes proved more easily) in…
In an earlier paper the authors proved that limits of convergent graph sequences can be described by various structures, including certain 2-variable real functions called graphons, random graph models satisfying certain consistency…
This paper presents a novel stochastic gradient descent algorithm for constrained optimization. The proposed algorithm randomly samples constraints and components of the finite sum objective function and relies on a relaxed logarithmic…
We develop the theory of discrete-time gradient flows for convex functions on Alexandrov spaces with arbitrary upper or lower curvature bounds. We employ different resolvent maps in the upper and lower curvature bound cases to construct…
A model for the evolution of a large population interacting system is considered in which a marked Poisson processes influences their evolution, together with a Brownian motion. Mean field McKean-Vlasov limits of such system are formulated…