Minimax Efficient Finite-Difference Stochastic Gradient Estimators Using Black-Box Function Evaluations
Statistics Theory
2020-11-13 v2 Methodology
Statistics Theory
Abstract
Standard approaches to stochastic gradient estimation, with only noisy black-box function evaluations, use the finite-difference method or its variants. While natural, it is open to our knowledge whether their statistical accuracy is the best possible. This paper argues so by showing that central finite-difference is a nearly minimax optimal zeroth-order gradient estimator for a suitable class of objective functions and mean squared risk, among both the class of linear estimators and the much larger class of all (nonlinear) estimators.
Cite
@article{arxiv.2007.04443,
title = {Minimax Efficient Finite-Difference Stochastic Gradient Estimators Using Black-Box Function Evaluations},
author = {Henry Lam and Haidong Li and Xuhui Zhang},
journal= {arXiv preprint arXiv:2007.04443},
year = {2020}
}