Contextual Bandits with Packing and Covering Constraints: A Modular Lagrangian Approach via Regression
Abstract
We consider contextual bandits with linear constraints (CBwLC), a variant of contextual bandits in which the algorithm consumes multiple resources subject to linear constraints on total consumption. This problem generalizes contextual bandits with knapsacks (CBwK), allowing for packing and covering constraints, as well as positive and negative resource consumption. We provide the first algorithm for CBwLC (or CBwK) that is based on regression oracles. The algorithm is simple, computationally efficient, and statistically optimal under mild assumptions. Further, we provide the first vanishing-regret guarantees for CBwLC (or CBwK) that extend beyond the stochastic environment. We side-step strong impossibility results from prior work by identifying a weaker (and, arguably, fairer) benchmark to compare against. Our algorithm builds on LagrangeBwK (Immorlica et al., FOCS 2019), a Lagrangian-based technique for CBwK, and SquareCB (Foster and Rakhlin, ICML 2020), a regression-based technique for contextual bandits. Our analysis leverages the inherent modularity of both techniques.
Keywords
Cite
@article{arxiv.2211.07484,
title = {Contextual Bandits with Packing and Covering Constraints: A Modular Lagrangian Approach via Regression},
author = {Aleksandrs Slivkins and Xingyu Zhou and Karthik Abinav Sankararaman and Dylan J. Foster},
journal= {arXiv preprint arXiv:2211.07484},
year = {2024}
}
Comments
A preliminary version of this paper, authored by A. Slivkins, K.A. Sankararaman and D.J. Foster, has been published at COLT 2023. The present version (since Jun'24) features an important improvement, due to Xingyu Zhou. The Oct'24 version fixes an inaccuracy in Section 6 when the analysis from Section 4 is invoked