ZeroShotOpt: Towards Zero-Shot Pretrained Models for Efficient Black-Box Optimization
Abstract
Global optimization of expensive, derivative-free black-box functions requires extreme sample efficiency. While Bayesian optimization (BO) is the current state-of-the-art, its performance hinges on surrogate and acquisition function hyper-parameters that are often hand-tuned and fail to generalize across problem landscapes. We present ZeroShotOpt, a general-purpose, pretrained model for continuous black-box optimization tasks ranging from 2D to 20D. Our approach leverages offline reinforcement learning on large-scale optimization trajectories collected from 12 BO variants. To scale pretraining, we generate millions of synthetic Gaussian process-based functions with diverse landscapes, enabling the model to learn transferable optimization policies. As a result, ZeroShotOpt achieves robust zero-shot generalization on a wide array of unseen benchmarks, matching or surpassing the sample efficiency of leading global optimizers, including BO, while also offering a reusable foundation for future extensions and improvements. Our open-source code, dataset, and model are available at: https://github.com/jamisonmeindl/zeroshotopt
Cite
@article{arxiv.2510.03051,
title = {ZeroShotOpt: Towards Zero-Shot Pretrained Models for Efficient Black-Box Optimization},
author = {Jamison Meindl and Yunsheng Tian and Tony Cui and Veronika Thost and Zhang-Wei Hong and Johannes Dürholt and Jie Chen and Wojciech Matusik and Mina Konaković Luković},
journal= {arXiv preprint arXiv:2510.03051},
year = {2025}
}