Adjusted Shuffling SARAH: Advancing Complexity Analysis via Dynamic Gradient Weighting
Abstract
In this paper, we propose Adjusted Shuffling SARAH, a novel algorithm that integrates shuffling strategies into the recursive SARAH framework using a dynamic weighting mechanism to enhance exploration. We analyze the algorithm under two operating modes. First, we show that the Exact Mode matches the best-known theoretical guarantees for shuffling variance-reduced methods in both strongly convex and non-convex settings. Second, to address large-scale regimes, we introduce an Inexact Mode that utilizes mini-batch estimators. A key contribution of our work is proving that this Inexact Mode achieves a total complexity independent of the dataset size, making it significantly more scalable than existing shuffling methods when the sample size is large.
Cite
@article{arxiv.2506.12444,
title = {Adjusted Shuffling SARAH: Advancing Complexity Analysis via Dynamic Gradient Weighting},
author = {Duc Toan Nguyen and Trang H. Tran and Lam M. Nguyen},
journal= {arXiv preprint arXiv:2506.12444},
year = {2026}
}