Hessian Inverse Approximation as Covariance for Random Perturbation in Black-Box Problems

In stochastic optimization problems using noisy zeroth-order (ZO) oracles only, the randomized counterpart of the Kiefer-Wolfowitz-type method is widely used to estimate the gradient. Existing algorithms generate randomized perturbation from a zero-mean and unit-covariance distribution. In contrast, this work considers the generalization where the perturbations have a possibly non-identity covariance constructed from the history of the ZO queries. We propose to feed the second-order approximation into the covariance matrix of the random perturbation, so it is dubbed as Hessian-aided random perturbation (HARP). HARP collects four zeroth-order queries per iteration to form approximations for both the gradient and the Hessian. We show the convergence (in an almost surely sense) and derive the convergence rate for HARP under mild assumptions. We demonstrate, with theoretical guarantees and numerical experiments, that HARP is less sensitive to ill-conditioning and more query-efficient than other gradient approximation schemes whose random perturbation has an identity covariance.