Related papers: Error distributions for random grid approximations…

We present a different view on stochastic optimization, which goes back to the splitting schemes for approximate solutions of ODE. In this work, we provide a connection between stochastic gradient descent approach and first-order splitting…

Loss functions with non-isolated minima have emerged in several machine learning problems, creating a gap between theory and practice. In this paper, we formulate a new type of local convexity condition that is suitable to describe the…

We study a filtered Lie splitting scheme for the cubic nonlinear Schr\"{o}dinger equation. We establish error estimates at low regularity by using discrete Bourgain spaces. This allows us to handle data in $H^s$ with $0<s<1$ overcoming the…

Necessary optimality conditions in Lagrangian form and the sequential minimization framework are extended to mixed-integer nonlinear optimization, without any convexity assumptions. Building upon a recently developed notion of local…

We study stochastic optimization of nonconvex loss functions, which are typical objectives for training neural networks. We propose stochastic approximation algorithms which optimize a series of regularized, nonlinearized losses on large…

We present an explicit method for simulating stochastic differential equations (SDEs) that have variable diffusion coefficients and satisfy the detailed balance condition with respect to a known equilibrium density. In Tupper and Yang…

We consider the problem of sampling from a target distribution, which is \emph {not necessarily logconcave}, in the context of empirical risk minimization and stochastic optimization as presented in Raginsky et al. (2017). Non-asymptotic…

We discuss conditions ensuring the (strict) convergence of stochastic gradient algorithms.

This work develops new results for stochastic approximation algorithms. The emphases are on treating algorithms and limits with discontinuities. The main ingredients include the use of differential inclusions, set-valued analysis, and…

We propose a general error analysis related to the low-rank approximation of a given real matrix in both the spectral and Frobenius norms. First, we derive deterministic error bounds that hold with some minimal assumptions. Second, we…

Online averaged stochastic gradient algorithms are more and more studied since (i) they can deal quickly with large sample taking values in high dimensional spaces, (ii) they enable to treat data sequentially, (iii) they are known to be…

We study the approximation of stochastic differential equations driven by a fractional Brownian motion with Hurst parameter $H>1/2$. For the mean-square error at a single point we derive the optimal rate of convergence that can be achieved…

Randomized algorithms have proven to perform well on a large class of numerical linear algebra problems. Their theoretical analysis is critical to provide guarantees on their behaviour, and in this sense, the stochastic analysis of the…

In the setting where we have $n$ independent observations of a random variable $X$, we derive explicit error bounds in total variation distance when approximating the number of observations equal to the maximum of the sample (in the case…

In this paper we present a convergence rate analysis of inexact variants of several randomized iterative methods. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic…

Wasserstein distributionally robust optimization offers a framework for model fitting in machine learning under potential shifts in the data distribution. We study a regularized variant of this problem in which entropic smoothing produces a…

In this paper, we propose a monotone approximation scheme for a class of fully nonlinear degenerate partial integro-differential equations (PIDEs) which characterize the nonlinear $\alpha$-stable L\'{e}vy processes under sublinear…

By combining a certain approximation property in the spatial domain, and weighted $\ell_2$-summability of the Hermite polynomial expansion coefficients in the parametric domain obtained in [M. Bachmayr, A. Cohen, R. DeVore and G.…

We present Selective Non-Gaussian Refinement (SNGR), a SLAM framework that augments iSAM2 with targeted nested sampling on windows where Gaussian approximations are likely to fail. We detect such regions using the condition number of joint…

Using lattice approximations of Euclidean space, we develop a way to approximate stable processes that are represented by stochastic integrals over Euclidean space. Via a stable version of the Lindeberg-Feller Theorem we show that the…