Online Semi-Supervised Learning in Contextual Bandits with Episodic Reward
Abstract
We considered a novel practical problem of online learning with episodically revealed rewards, motivated by several real-world applications, where the contexts are nonstationary over different episodes and the reward feedbacks are not always available to the decision making agents. For this online semi-supervised learning setting, we introduced Background Episodic Reward LinUCB (BerlinUCB), a solution that easily incorporates clustering as a self-supervision module to provide useful side information when rewards are not observed. Our experiments on a variety of datasets, both in stationary and nonstationary environments of six different scenarios, demonstrated clear advantages of the proposed approach over the standard contextual bandit. Lastly, we introduced a relevant real-life example where this problem setting is especially useful.
Keywords
Cite
@article{arxiv.2009.08457,
title = {Online Semi-Supervised Learning in Contextual Bandits with Episodic Reward},
author = {Baihan Lin},
journal= {arXiv preprint arXiv:2009.08457},
year = {2020}
}
Comments
Proceeding of AJCAI 2020. This article supersedes our work arXiv:1802.00981 on contextual bandits in nonstationary setting, introduces a new problem setting with episodically revealed reward, and provides a novel solution by propagating pseudo-feedbacks to un-rewarded cases from self-supervision. Also check out our speaker diarization application of this algorithm at arXiv:2006.04376