Gradient-Free Methods for Non-Smooth Convex Stochastic Optimization with Heavy-Tailed Noise on Convex Compact
Optimization and Control
2023-08-25 v3
Abstract
We present two easy-to-implement gradient-free/zeroth-order methods to optimize a stochastic non-smooth function accessible only via a black-box. The methods are built upon efficient first-order methods in the heavy-tailed case, i.e., when the gradient noise has infinite variance but bounded -th moment for some . The first algorithm is based on the stochastic mirror descent with a particular class of uniformly convex mirror maps which is robust to heavy-tailed noise. The second algorithm is based on the stochastic mirror descent and gradient clipping technique. Additionally, for the objective functions satisfying the -growth condition, faster algorithms are proposed based on these methods and the restart technique.
Cite
@article{arxiv.2304.02442,
title = {Gradient-Free Methods for Non-Smooth Convex Stochastic Optimization with Heavy-Tailed Noise on Convex Compact},
author = {Nikita Kornilov and Alexander Gasnikov and Pavel Dvurechensky and Darina Dvinskikh},
journal= {arXiv preprint arXiv:2304.02442},
year = {2023}
}