Related papers: Ergodic convergence of a stochastic proximal point…
We generalize stochastic subgradient descent methods to situations in which we do not receive independent samples from the distribution over which we optimize, but instead receive samples that are coupled over time. We show that as long as…
This paper considers a stochastic optimization problem over the fixed point sets of quasinonexpansive mappings on Riemannian manifolds. The problem enables us to consider Riemannian hierarchical optimization problems over complicated sets,…
We establish weak-type $(1,1)$ bounds for the maximal function associated with ergodic averaging operators modeled on a wide class of thin deterministic sets $B$. As a corollary we obtain the corresponding pointwise convergence result on…
Optimization problems with access to only zeroth-order information of the objective function on Riemannian manifolds arise in various applications, spanning from statistical learning to robot learning. While various zeroth-order algorithms…
In this paper we consider convex optimization problems with stochastic composite objective function subject to (possibly) infinite intersection of constraints. The objective function is expressed in terms of expectation operator over a sum…
Since introduced by Martinet and Rockafellar, the proximal point algorithm was generalized in many fruitful directions. More recently, in 2002, Pennanen studied the proximal point algorithm without monotonicity. A year later, Iusem and…
We study the weak convergence of iterates of so-called centred kernel quadratic stochastic operators. These iterations, in a population evolution setting, describe the additive perturbation of the arithmetic mean of the traits of an…
We propose an algorithm for approximating the solution of a strongly oscillating SDE, that is, a system in which some ergodic state variables evolve quickly with respect to the other variables. The algorithm profits from homogenization…
This work develops new results for stochastic approximation algorithms. The emphases are on treating algorithms and limits with discontinuities. The main ingredients include the use of differential inclusions, set-valued analysis, and…
Stochastic optimization lies at the core of most statistical learning models. The recent great development of stochastic algorithmic tools focused significantly onto proximal gradient iterations, in order to find an efficient approach for…
Stochastic gradient descent type methods are ubiquitous in machine learning, but they are only applicable to the optimization of differentiable functions. Proximal algorithms are more general and applicable to nonsmooth functions. We…
Motivated by recent work on stochastic gradient descent methods, we develop two stochastic variants of greedy algorithms for possibly non-convex optimization problems with sparsity constraints. We prove linear convergence in expectation to…
Stochastic non-smooth convex optimization constitutes a class of problems in machine learning and operations research. This paper considers minimization of a non-smooth function based on stochastic subgradients. When the function has a…
Given a strictly convex multiobjective optimization problem with objective functions $f_1,\dots,f_N$, let us denote by $x_0$ its solution, obtained as minimum point of the linear scalarized problem, where the objective function is the…
We present a simple randomized procedure for the prediction of a binary sequence. The algorithm uses ideas from recent developments of the theory of the prediction of individual sequences. We show that if the sequence is a realization of a…
We analyze a stochastic approximation algorithm for decision-dependent problems, wherein the data distribution used by the algorithm evolves along the iterate sequence. The primary examples of such problems appear in performative prediction…
In this paper, we design an inertial accelerated primal-dual algorithm to address the convex-concave saddle point problem, which is formulated as $\min_{x}\max_{y} f(x) + \langle Kx, y \rangle - g(y)$. Remarkably, both functions $f$ and $g$…
A sequential quadratic programming method is designed for solving general smooth nonlinear stochastic optimization problems subject to expectation equality constraints. We consider the setting where the objective and constraint function…
In [19], a general, inexact, efficient proximal quasi-Newton algorithm for composite optimization problems has been proposed and a sublinear global convergence rate has been established. In this paper, we analyze the convergence properties…
The ODE method has been a workhorse for algorithm design and analysis since the introduction of the stochastic approximation. It is now understood that convergence theory amounts to establishing robustness of Euler approximations for ODEs,…