English

Gradient-Variation Regret Bounds for Unconstrained Online Learning

Machine Learning 2026-04-14 v1 Machine Learning

Abstract

We develop parameter-free algorithms for unconstrained online learning with regret guarantees that scale with the gradient variation VT(u)=t=2Tft(u)ft1(u)2V_T(u) = \sum_{t=2}^T \|\nabla f_t(u)-\nabla f_{t-1}(u)\|^2. For LL-smooth convex loss, we provide fully-adaptive algorithms achieving regret of order O~(uVT(u)+Lu2+G4)\widetilde{O}(\|u\|\sqrt{V_T(u)} + L\|u\|^2+G^4) without requiring prior knowledge of comparator norm u\|u\|, Lipschitz constant GG, or smoothness LL. The update in each round can be computed efficiently via a closed-form expression. Our results extend to dynamic regret and find immediate implications to the stochastically-extended adversarial (SEA) model, which significantly improves upon the previous best-known result [Wang et al., 2025].

Keywords

Cite

@article{arxiv.2604.11151,
  title  = {Gradient-Variation Regret Bounds for Unconstrained Online Learning},
  author = {Yuheng Zhao and Andrew Jacobsen and Nicolò Cesa-Bianchi and Peng Zhao},
  journal= {arXiv preprint arXiv:2604.11151},
  year   = {2026}
}
R2 v1 2026-07-01T12:05:51.904Z