Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
Abstract
We present global convergence rates for a line-search method which is based on random first-order models and directions whose quality is ensured only with certain probability. We show that in terms of the order of the accuracy, the evaluation complexity of such a method is the same as its counterparts that use deterministic accurate models; the use of probabilistic models only increases the complexity by a constant, which depends on the probability of the models being good. We particularize and improve these results in the convex and strongly convex case. We also analyze a probabilistic cubic regularization variant that allows approximate probabilistic second-order models and show improved complexity bounds compared to probabilistic first-order methods; again, as a function of the accuracy, the probabilistic cubic regularization bounds are of the same (optimal) order as for the deterministic case.
Cite
@article{arxiv.1505.06070,
title = {Global convergence rate analysis of unconstrained optimization methods based on probabilistic models},
author = {Coralia Cartis and Katya Scheinberg},
journal= {arXiv preprint arXiv:1505.06070},
year = {2017}
}