Related papers: Stochastic optimization on matrices and a graphon …
We study stochastic gradient descent (SGD) for composite optimization problems with $N$ sequential operators subject to perturbations in both the forward and backward passes. Unlike classical analyses that treat gradient noise as additive…
Realistic networks display heterogeneous transmission delays. We analyze here the limits of large stochastic multi-populations networks with stochastic coupling and random interconnection delays. We show that depending on the nature of the…
We study a system of reflected Brownian motions on the positive half-line in which each particle has a drift toward the origin determined by the local times at the origin of all the particles. If this local time drift is too strong, such…
It seems that in the current age, computers, computation, and data have an increasingly important role to play in scientific research and discovery. This is reflected in part by the rise of machine learning and artificial intelligence,…
Consider a system of $n$ weakly interacting particles driven by independent Brownian motions. In many instances, it is well known that the empirical measure converges to the solution of a partial differential equation, usually called…
In this paper we study the asymptotic behavior of stochastic approximation schemes with set-valued drift function and non-additive iterate-dependent Markov noise. We show that a linearly interpolated trajectory of such a recursion is an…
Stochastic gradient descent (SGD) is central to simulation optimization, stochastic programming, and online M-estimation, where sampling effort is a decision variable. We study the mini-batch gradient noise as a sampling-design object.…
Stochastic constraints, which incorporate both deterministic parameters and random variables, extend classical deterministic constraints by explicitly accounting for uncertainty. These constraints are increasingly prevalent in data science,…
Understanding the behavior of stochastic gradient methods is a central problem in modern machine learning. Recent work has highlighted diagonal linear networks as a simplified yet expressive setting for analyzing the optimization and…
The concept of a random process has been recently extended to graph signals, whereby random graph processes are a class of multivariate stochastic processes whose coefficients are matrices with a \textit{graph-topological} structure. The…
Graph convolutional networks (GCNs) are a widely used method for graph representation learning. To elucidate the capabilities and limitations of GCNs, we investigate their power, as a function of their number of layers, to distinguish…
We consider a general McKean-Vlasov stochastic differential equation driven by a rotationally invariant $\alpha$-stable process on $\mathbb{R}^d$ with $\alpha \in (1,2)$. We assume that the diffusion coefficient is the identity matrix and…
A large deviation principle is established for a general class of stochastic flows in the small noise limit. This result is then applied to a Bayesian formulation of an image matching problem, and an approximate maximum likelihood property…
This paper is devoted to the investigation of gradient flows in asymmetric metric spaces (for example, irreversible Finsler manifolds and Minkowski normed spaces) by means of discrete approximation. We study basic properties of curves and…
Dimensional reduction techniques have long been used to visualize the structure and geometry of high dimensional data. However, most widely used techniques are difficult to interpret due to nonlinearities and opaque optimization processes.…
In [Athreya, den Hollander, R\"ollin; 2021, arXiv:1908.06241] models from population genetics were used to define stochastic dynamics in the space of graphons arising as continuum limits of dense graphs. In the present paper we exhibit an…
A large deviation principle is established for a two-scale stochastic system in which the slow component is a continuous process given by a small noise finite dimensional It\^{o} stochastic differential equation, and the fast component is a…
We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the…
Subspace learning and matrix factorization problems have great many applications in science and engineering, and efficient algorithms are critical as dataset sizes continue to grow. Many relevant problem formulations are non-convex, and in…
We provide an explicit rigorous derivation of a diffusion limit - a stochastic differential equation with additive noise - from a deterministic skew-product flow. This flow is assumed to exhibit time-scale separation and has the form of a…