Black-Box Policy Search with Probabilistic Programs
Machine Learning
2016-08-05 v4 Artificial Intelligence
Abstract
In this work, we explore how probabilistic programs can be used to represent policies in sequential decision problems. In this formulation, a probabilistic program is a black-box stochastic simulator for both the problem domain and the agent. We relate classic policy gradient techniques to recently introduced black-box variational methods which generalize to probabilistic program inference. We present case studies in the Canadian traveler problem, Rock Sample, and a benchmark for optimal diagnosis inspired by Guess Who. Each study illustrates how programs can efficiently represent policies using moderate numbers of parameters.
Cite
@article{arxiv.1507.04635,
title = {Black-Box Policy Search with Probabilistic Programs},
author = {Jan-Willem van de Meent and Brooks Paige and David Tolpin and Frank Wood},
journal= {arXiv preprint arXiv:1507.04635},
year = {2016}
}