An investigation of stochastic trust-region based algorithms for finite-sum minimization
Optimization and Control
2024-04-23 v1
Abstract
This work elaborates on the TRust-region-ish (TRish) algorithm, a stochastic optimization method for finite-sum minimization problems proposed by Curtis et al. in [Curtis2019, Curtis2022]. A theoretical analysis that complements the results in the literature is presented, and the issue of tuning the involved hyper-parameters is investigated. Our study also focuses on a practical version of the method, which computes the stochastic gradient by means of the inner product test and the orthogonality test proposed by Bollapragada et al. in [Bollapragada2018]. It is shown experimentally that this implementation improves the performance of TRish and reduces its sensitivity to the choice of the hyper-parameters.
Keywords
Cite
@article{arxiv.2404.13382,
title = {An investigation of stochastic trust-region based algorithms for finite-sum minimization},
author = {Stefania Bellavia and Benedetta Morini and Simone Rebegoldi},
journal= {arXiv preprint arXiv:2404.13382},
year = {2024}
}