Stochastic Optimization with Random Search
Optimization and Control
2025-12-19 v2 Machine Learning
Abstract
We revisit random search for stochastic optimization, where only noisy function evaluations are available. We show that the method works under weaker smoothness assumptions than previously considered, and that stronger assumptions enable improved guarantees. In the finite-sum setting, we design a variance-reduced variant that leverages multiple samples to accelerate convergence. Our analysis relies on a simple translation invariance property, which provides a principled way to balance noise and reduce variance.
Cite
@article{arxiv.2510.15610,
title = {Stochastic Optimization with Random Search},
author = {El Mahdi Chayti and Taha El Bakkali El Kadi and Omar Saadi and Martin Jaggi},
journal= {arXiv preprint arXiv:2510.15610},
year = {2025}
}