English

Heterogeneous-Horizon Exact-Weight Local SGD

Optimization and Control 2026-04-29 v2

Abstract

We study adaptive aggregation for heterogeneous local SGD in convex finite-sum optimization, allowing heterogeneous local horizons, minibatch sizes, gradient noise, and participation. We introduce HEW-Local SGD, a corrected local-SGD method that chooses nodewise server weights by minimizing an explicit one-round upper bound on the next objective value. This yields an exact local-control formulation with a threshold simplex update, separable amplitude updates, and a one-step guarantee under arbitrary predictable participation. We also introduce two post-local variants: a corrected heterogeneous method and a simpler homogeneous specialization. We establish one-step guarantees and global benchmark-style convergence results. In the regimes where comparison is appropriate, the theory matches the qualitative communication-efficient picture of recent LocalSGD/SCAFFOLD analyses, while also giving explicit guarantees for unequal local horizons.

Keywords

Cite

@article{arxiv.2604.24463,
  title  = {Heterogeneous-Horizon Exact-Weight Local SGD},
  author = {Dmitry Pasechnyuk-Vilensky and Martin Takáč},
  journal= {arXiv preprint arXiv:2604.24463},
  year   = {2026}
}

Comments

68 pages

R2 v1 2026-07-01T12:37:13.492Z