Adaptive State-Dependent Diffusion for Derivative-Free Optimization
Optimization and Control
2023-02-10 v1 Machine Learning
Numerical Analysis
Numerical Analysis
Abstract
This paper develops and analyzes a stochastic derivative-free optimization strategy. A key feature is the state-dependent adaptive variance. We prove global convergence in probability with algebraic rate and give the quantitative results in numerical examples. A striking fact is that convergence is achieved without explicit information of the gradient and even without comparing different objective function values as in established methods such as the simplex method and simulated annealing. It can otherwise be compared to annealing with state-dependent temperature.
Cite
@article{arxiv.2302.04370,
title = {Adaptive State-Dependent Diffusion for Derivative-Free Optimization},
author = {Björn Engquist and Kui Ren and Yunan Yang},
journal= {arXiv preprint arXiv:2302.04370},
year = {2023}
}