Sample Average Approximation for Black-Box VI
Machine Learning
2023-05-18 v2 Optimization and Control
Machine Learning
Abstract
We present a novel approach for black-box VI that bypasses the difficulties of stochastic gradient ascent, including the task of selecting step-sizes. Our approach involves using a sequence of sample average approximation (SAA) problems. SAA approximates the solution of stochastic optimization problems by transforming them into deterministic ones. We use quasi-Newton methods and line search to solve each deterministic optimization problem and present a heuristic policy to automate hyperparameter selection. Our experiments show that our method simplifies the VI problem and achieves faster performance than existing methods.
Keywords
Cite
@article{arxiv.2304.06803,
title = {Sample Average Approximation for Black-Box VI},
author = {Javier Burroni and Justin Domke and Daniel Sheldon},
journal= {arXiv preprint arXiv:2304.06803},
year = {2023}
}