Related papers: Convergence guarantees for stochastic algorithms s…
This paper is a continuation to the study of generalized quasi contractive operators, essentially due to Akhtar et al. [A multi-step implicit iterative process for common fixed points of generalized C^{q}-operators in convex metric spaces,…
During recent years the interest of optimization and machine learning communities in high-probability convergence of stochastic optimization methods has been growing. One of the main reasons for this is that high-probability complexity…
We establish presumably optimal rates of normal convergence with respect to the Kolmogorov distance for a large class of geometric functionals of marked Poisson and binomial point processes on general metric spaces. The rates are valid…
We revisit the sample average approximation (SAA) approach for non-convex stochastic programming. We show that applying the SAA approach to problems with expected value equality constraints does not necessarily result in asymptotic…
Stochastic gradient optimization methods are broadly used to minimize non-convex smooth objective functions, for instance when training deep neural networks. However, theoretical guarantees on the asymptotic behaviour of these methods…
This paper examines the asymptotic convergence properties of Lipschitz interpolation methods within the context of bounded stochastic noise. In the first part of the paper, we establish probabilistic consistency guarantees of the classical…
This paper considers stochastic optimization problems for a large class of objective functions, including convex and continuous submodular. Stochastic proximal gradient methods have been widely used to solve such problems; however, their…
We introduce the notion of a random relaxed asymptotic contraction in the setting of random normed modules. The contraction condition employs two quasi-metrics that are built directly from the random operator: a lower quasi-metric which…
In this paper, we aim to study the asymptotic behaviour for a class of McKean-Vlasov stochastic partial differential equations with slow and fast time-scales. Using the variational approach and classical Khasminskii time discretization, we…
In this paper, we study the convergence properties of the Stochastic Gradient Descent (SGD) method for finding a stationary point of a given objective function $J(\cdot)$. The objective function is not required to be convex. Rather, our…
During the last years, asymptotic (or sequential) constraint qualifications, which postulate upper semicontinuity of certain set-valued mappings and provide a natural companion of asymptotic stationarity conditions, have been shown to be…
In this paper, we study the almost sure boundedness and the convergence of the stochastic approximation (SA) algorithm. At present, most available convergence proofs are based on the ODE method, and the almost sure boundedness of the…
Motivated by applications in optimization and machine learning, we consider stochastic quasi-Newton (SQN) methods for solving stochastic optimization problems. In the literature, the convergence analysis of these algorithms relies on strong…
We consider a family of parallel methods for constrained optimization based on projected gradient descents along individual coordinate directions. In the case of polyhedral feasible sets, local convergence towards a regular solution occurs…
In this work, we investigate stochastic quasi-Newton methods for minimizing a finite sum of cost functions over a decentralized network. In Part I, we develop a general algorithmic framework that incorporates stochastic quasi-Newton…
Building upon the technique that we developed earlier for perturbed sweeping processes with convex moving constraints and monotone vector fields (Kamenskii et al, Nonlinear Anal. Hybrid Syst. 30, 2018), the present paper establishes global…
Classical convergence analyses for optimization algorithms rely on the widely-adopted uniform smoothness assumption. However, recent experimental studies have demonstrated that many machine learning problems exhibit non-uniform smoothness,…
Cubic-regularized Newton's method (CR) is a popular algorithm that guarantees to produce a second-order stationary solution for solving nonconvex optimization problems. However, existing understandings of the convergence rate of CR are…
We undertake a precise study of the asymptotic and non-asymptotic properties of stochastic approximation procedures with Polyak-Ruppert averaging for solving a linear system $\bar{A} \theta = \bar{b}$. When the matrix $\bar{A}$ is Hurwitz,…
Optimization is important in machine learning problems, and quasi-Newton methods have a reputation as the most efficient numerical schemes for smooth unconstrained optimization. In this paper, we consider the explicit superlinear…