Complexity and performance for two classes of noise-tolerant first-order algorithms
Optimization and Control
2025-01-30 v3
Abstract
Two classes of algorithms for optimization in the presence of noise are presented, that do not require the evaluation of the objective function. The first generalizes the well-known Adagrad method. Its complexity is then analyzed as a function of its parameters. A second class of algorithms is then derived whose complexity is at least as good as that of the first class. Initial numerical experiments on finite-sum problems arising from deep-learning applications suggest that methods of the second class may outperform those of the first.
Cite
@article{arxiv.2203.01757,
title = {Complexity and performance for two classes of noise-tolerant first-order algorithms},
author = {S. Gratton and S. Jerad and Ph. L. Toint},
journal= {arXiv preprint arXiv:2203.01757},
year = {2025}
}
Comments
3 figures. arXiv admin note: substantial text overlap with arXiv:2203.01647