Conditional Importance Sampling for Off-Policy Learning
Machine Learning
2020-07-31 v2 Machine Learning
Abstract
The principal contribution of this paper is a conceptual framework for off-policy reinforcement learning, based on conditional expectations of importance sampling ratios. This framework yields new perspectives and understanding of existing off-policy algorithms, and reveals a broad space of unexplored algorithms. We theoretically analyse this space, and concretely investigate several algorithms that arise from this framework.
Cite
@article{arxiv.1910.07479,
title = {Conditional Importance Sampling for Off-Policy Learning},
author = {Mark Rowland and Anna Harutyunyan and Hado van Hasselt and Diana Borsa and Tom Schaul and Rémi Munos and Will Dabney},
journal= {arXiv preprint arXiv:1910.07479},
year = {2020}
}
Comments
AISTATS 2020 camera-ready version