Minibatch Stochastic Three Points Method for Unconstrained Smooth Minimization
Optimization and Control
2022-09-19 v1 Machine Learning
Abstract
In this paper, we propose a new zero order optimization method called minibatch stochastic three points (MiSTP) method to solve an unconstrained minimization problem in a setting where only an approximation of the objective function evaluation is possible. It is based on the recently proposed stochastic three points (STP) method (Bergou et al., 2020). At each iteration, MiSTP generates a random search direction in a similar manner to STP, but chooses the next iterate based solely on the approximation of the objective function rather than its exact evaluations. We also analyze our method's complexity in the nonconvex and convex cases and evaluate its performance on multiple machine learning tasks.
Cite
@article{arxiv.2209.07883,
title = {Minibatch Stochastic Three Points Method for Unconstrained Smooth Minimization},
author = {Soumia Boucherouite and Grigory Malinovsky and Peter Richtárik and EL Houcine Bergou},
journal= {arXiv preprint arXiv:2209.07883},
year = {2022}
}