English

Stochastic continuum armed bandit problem of few linear parameters in high dimensions

Machine Learning 2017-05-31 v4 Machine Learning Optimization and Control

Abstract

We consider a stochastic continuum armed bandit problem where the arms are indexed by the 2\ell_2 ball Bd(1+ν)B_{d}(1+\nu) of radius 1+ν1+\nu in Rd\mathbb{R}^d. The reward functions r:Bd(1+ν)Rr :B_{d}(1+\nu) \rightarrow \mathbb{R} are considered to intrinsically depend on kdk \ll d unknown linear parameters so that r(x)=g(Ax)r(\mathbf{x}) = g(\mathbf{A} \mathbf{x}) where A\mathbf{A} is a full rank k×dk \times d matrix. Assuming the mean reward function to be smooth we make use of results from low-rank matrix recovery literature and derive an efficient randomized algorithm which achieves a regret bound of O(C(k,d)n1+k2+k(logn)12+k)O(C(k,d) n^{\frac{1+k}{2+k}} (\log n)^{\frac{1}{2+k}}) with high probability. Here C(k,d)C(k,d) is at most polynomial in dd and kk and nn is the number of rounds or the sampling budget which is assumed to be known beforehand.

Keywords

Cite

@article{arxiv.1312.0232,
  title  = {Stochastic continuum armed bandit problem of few linear parameters in high dimensions},
  author = {Hemant Tyagi and Sebastian Stich and Bernd Gärtner},
  journal= {arXiv preprint arXiv:1312.0232},
  year   = {2017}
}

Comments

Changes from previous version: (a) Corrected typos throughout. (b) In earlier version, regret was defined as a conditional expectation (and hence bounded w.h.p); this is changed to an expectation now resulting in minor changes in statements of Lemma 1, Theorems 1,2 and Corollary 1. See Remark 1. (c) Added Remark 3, and corrected statement of Proposition 3

R2 v1 2026-06-22T02:18:23.359Z