Stochastic ADMM with batch size adaptation for nonconvex nonsmooth optimization
Abstract
Stochastic alternating direction method of multipliers (SADMM) is a popular method for solving nonconvex nonsmooth optimization in various applications. However, it typically requires an empirical selection of the static batch size for gradient estimation, resulting in a challenging trade-off between variance reduction and computational cost. This paper proposes adaptive batch size SADMM, a practical method that dynamically adjusts the batch size based on accumulated differences along the optimization path. We develop a simple convergence analysis to handle the dependence of batch size adaptation that matches the best-known complexity with flexible parameter choices. We further extend this adaptive scheme to reduce the overall complexity of the popular variance-reduced methods, SVRG-ADMM and SPIDER-ADMM. Numerical results validate the effectiveness of our proposed methods.
Keywords
Cite
@article{arxiv.2505.06921,
title = {Stochastic ADMM with batch size adaptation for nonconvex nonsmooth optimization},
author = {Jiachen Jin and Kangkang Deng and Boyu Wang and Hongxia Wang},
journal= {arXiv preprint arXiv:2505.06921},
year = {2026}
}