Thompson Sampling: An Asymptotically Optimal Finite Time Analysis
Machine Learning
2012-07-20 v2 Machine Learning
Abstract
The question of the optimality of Thompson Sampling for solving the stochastic multi-armed bandit problem had been open since 1933. In this paper we answer it positively for the case of Bernoulli rewards by providing the first finite-time analysis that matches the asymptotic rate given in the Lai and Robbins lower bound for the cumulative regret. The proof is accompanied by a numerical comparison with other optimal policies, experiments that have been lacking in the literature until now for the Bernoulli case.
Cite
@article{arxiv.1205.4217,
title = {Thompson Sampling: An Asymptotically Optimal Finite Time Analysis},
author = {Emilie Kaufmann and Nathaniel Korda and Rémi Munos},
journal= {arXiv preprint arXiv:1205.4217},
year = {2012}
}
Comments
15 pages, 2 figures, submitted to ALT (Algorithmic Learning Theory)