Related papers: Stochastic optimization on matrices and a graphon …

For optimizing a non-convex function in finite dimension, a method is to add Brownian noise to a gradient descent, allowing for transitions between basins of attractions of different minimizers. To adapt this for optimization over a space…

Stochastic gradient optimization is the dominant learning paradigm for a variety of scenarios, from classical supervised learning to modern self-supervised learning. We consider stochastic gradient algorithms for learning problems whose…

We highlight a topological aspect of the graph limit theory. Graphons are limit objects for convergent sequences of dense graphs. We introduce the representation of a graphon on a unique metric space and we relate the dimension of this…

Stochastic differential equations of Langevin-diffusion form have received significant attention, thanks to their foundational role in both Bayesian sampling algorithms and optimization in machine learning. In the latter, they serve as a…

We study a variation of vanilla stochastic gradient descent where the optimizer only has access to a Markovian sampling scheme. These schemes encompass applications that range from decentralized optimization with a random walker (token…

We provide tight finite-time convergence bounds for gradient descent and stochastic gradient descent on quadratic functions, when the gradients are delayed and reflect iterates from $\tau$ rounds ago. First, we show that without stochastic…

The goal of this work is to identify steady-state solutions to dynamical systems defined on large, random families of networks. We do so by passing to a continuum limit where the adjacency matrix is replaced by a non-local operator with…

Consider an interacting particle system indexed by the vertices of a (possibly random) locally finite graph whose vertices and edges are equipped with marks representing parameters of the model such as the environment and initial…

We introduce a new notion of graph sparsificaiton based on spectral similarity of graph Laplacians: spectral sparsification requires that the Laplacian quadratic form of the sparsifier approximate that of the original. This is equivalent to…

We derive conditions on the products of stochastic matrices guaranteeing the existence of a unique limit invariant distribution. Belying our approach is the hereby defined notion of restricted triangulated Laman graphs. The main idea is the…

In this paper, we propose a scalable algorithm for spectral embedding. The latter is a standard tool for graph clustering. However, its computational bottleneck is the eigendecomposition of the graph Laplacian matrix, which prevents its…

Graph-limit theory focuses on the convergence of sequences of graphs when the number of nodes becomes arbitrarily large. This framework defines a continuous version of graphs allowing for the study of dynamical systems on very large graphs,…

We consider a family of random trees satisfying a Markov branching property. Roughly, this property says that the subtrees above some given height are independent with a law that depends only on their total size, the latter being either the…

In view of solving convex optimization problems with noisy gradient input, we analyze the asymptotic behavior of gradient-like flows under stochastic disturbances. Specifically, we focus on the widely studied class of mirror descent schemes…

This paper considers stochastic-constrained stochastic optimization where the stochastic constraint is to satisfy that the expectation of a random function is below a certain threshold. In particular, we study the setting where data samples…

This work studies constrained stochastic optimization problems where the objective and constraint functions are convex and expressed as compositions of stochastic functions. The problem arises in the context of fair classification, fair…

We consider the setting of multiscale overdamped Langevin stochastic differential equations, and study the problem of learning the drift function of the homogenized dynamics from continuous-time observations of the multiscale system. We…

We study the discrete-to-continuum consistency of the training of shallow graph convolutional neural networks (GCNNs) on proximity graphs of sampled point clouds under a manifold assumption. Graph convolution is defined spectrally via the…

In this paper we study the asymptotic behavior of a stochastic approximation scheme on two timescales with set-valued drift functions and in the presence of non-additive iterate-dependent Markov noise. It is shown that the recursion on each…

This study deals with continuous limits of interacting one-dimensional diffusive systems, arising from stochastic distortions of discrete curves with various kinds of coding representations. These systems are essentially of a…