Distributed Cooperative Decision-Making in Multiarmed Bandits: Frequentist and Bayesian Algorithms
Abstract
We study distributed cooperative decision-making under the explore-exploit tradeoff in the multiarmed bandit (MAB) problem. We extend the state-of-the-art frequentist and Bayesian algorithms for single-agent MAB problems to cooperative distributed algorithms for multi-agent MAB problems in which agents communicate according to a fixed network graph. We rely on a running consensus algorithm for each agent's estimation of mean rewards from its own rewards and the estimated rewards of its neighbors. We prove the performance of these algorithms and show that they asymptotically recover the performance of a centralized agent. Further, we rigorously characterize the influence of the communication graph structure on the decision-making performance of the group.
Cite
@article{arxiv.1606.00911,
title = {Distributed Cooperative Decision-Making in Multiarmed Bandits: Frequentist and Bayesian Algorithms},
author = {Peter Landgren and Vaibhav Srivastava and Naomi Ehrich Leonard},
journal= {arXiv preprint arXiv:1606.00911},
year = {2019}
}
Comments
This revision provides a correction to the original paper, which appeared in the Proceedings of the 2016 IEEE Conference on Decision and Control (CDC). The second statement of Proposition 1 and Theorem 1 are new from arXiv:1512.06888v3 and Lemma 1 is new. These are used to prove regret bounds in Theorems 2 and 3