A Unified Inexact Stochastic ADMM for Composite Nonconvex and Nonsmooth Optimization
Abstract
In this paper, we propose a unified framework of inexact stochastic Alternating Direction Method of Multipliers (ADMM) for solving nonconvex problems subject to linear constraints, whose objective comprises an average of finite-sum smooth functions and a nonsmooth but possibly nonconvex function. The new framework is highly versatile. Firstly, it not only covers several existing algorithms such as SADMM, SVRG-ADMM, and SPIDER-ADMM but also guides us to design a novel accelerated hybrid stochastic ADMM algorithm, which utilizes a new hybrid estimator to trade-off variance and bias. Second, it enables us to exploit a more flexible dual stepsize in the convergence analysis. Under some mild conditions, our unified framework preserves sublinear convergence. Additionally, we establish the linear convergence under error bound conditions. Finally, numerical experiments demonstrate the efficacy of the new algorithm for some nonsmooth and nonconvex problems.
Cite
@article{arxiv.2403.02015,
title = {A Unified Inexact Stochastic ADMM for Composite Nonconvex and Nonsmooth Optimization},
author = {Yuxuan Zeng and Jianchao Bai and Shengjia Wang and Zhiguo Wang},
journal= {arXiv preprint arXiv:2403.02015},
year = {2024}
}