An Element-wise RSAV Algorithm for Unconstrained Optimization Problems
Optimization and Control
2023-09-11 v1 Machine Learning
Machine Learning
Abstract
We present a novel optimization algorithm, element-wise relaxed scalar auxiliary variable (E-RSAV), that satisfies an unconditional energy dissipation law and exhibits improved alignment between the modified and the original energy. Our algorithm features rigorous proofs of linear convergence in the convex setting. Furthermore, we present a simple accelerated algorithm that improves the linear convergence rate to super-linear in the univariate case. We also propose an adaptive version of E-RSAV with Steffensen step size. We validate the robustness and fast convergence of our algorithm through ample numerical experiments.
Cite
@article{arxiv.2309.04013,
title = {An Element-wise RSAV Algorithm for Unconstrained Optimization Problems},
author = {Shiheng Zhang and Jiahao Zhang and Jie Shen and Guang Lin},
journal= {arXiv preprint arXiv:2309.04013},
year = {2023}
}
Comments
25 pages, 7 figures