Stochastic ISTA/FISTA Adaptive Step Search Algorithms for Convex Composite Optimization
Optimization and Control
2025-02-14 v2
Abstract
We develop and analyze stochastic variants of ISTA and a full backtracking FISTA algorithms [Beck and Teboulle, 2009, Scheinberg et al., 2014] for composite optimization without the assumption that stochastic gradient is an unbiased estimator. This work extends analysis of inexact fixed step ISTA/FISTA in [Schmidt et al., 2011] to the case of stochastic gradient estimates and adaptive step-size parameter chosen by backtracking. It also extends the framework for analyzing stochastic line-search method in [Cartis and Scheinberg, 2018] to the proximal gradient framework as well as to the accelerated first order methods.
Cite
@article{arxiv.2402.15646,
title = {Stochastic ISTA/FISTA Adaptive Step Search Algorithms for Convex Composite Optimization},
author = {Lam M. Nguyen and Katya Scheinberg and Trang H. Tran},
journal= {arXiv preprint arXiv:2402.15646},
year = {2025}
}
Comments
To appear at the Journal of Optimization Theory and Applications