English

Cubic Regularized Newton Method with Variance Reduction for Finite-sum Non-convex Problems

Optimization and Control 2026-04-28 v2

Abstract

We study finite-sum non-convex optimization minxRdF(x)  =  1ni=1nfi(x)\min_{x\in\mathbb{R}^d} F(x) \;=\; \frac{1}{n}\sum_{i=1}^n f_i(x) and analyze a variance-reduced cubic Newton method based on EMA-smoothed SARAH estimators for both gradient and Hessian information. The method combines a coarse stochastic backbone with a terminal homotopy refinement: once the iterates enter a certified small-step regime, the algorithm decreases the regularization level geometrically, shortens the stage length, and increases the mini-batch size at the reciprocal rate while restarting exact finite-sum snapshots at each stage. We work under average squared gradient smoothness and average mean-cubed Hessian smoothness, thereby avoiding the trajectory-wise Hessian boundedness assumption that is often used in related analyses. Under these assumptions and a standard inexact cubic-subproblem certificate, we establish that the method returns an (ε,L2ε)(\varepsilon,\sqrt{L_2\varepsilon})-second-order stationary point with total finite-sum oracle complexity n+O~ ⁣(n1/2ε3/2)n+\widetilde{\mathcal O}\!\left(n^{1/2}\varepsilon^{-3/2}\right). The analysis separates into a coarse progress phase, which yields the n1/2n^{1/2}-scaled stochastic backbone, and a terminal local bootstrap, which supplies the pointwise accuracy needed to turn the model step certificate into a true second-order certificate.

Keywords

Cite

@article{arxiv.2510.08714,
  title  = {Cubic Regularized Newton Method with Variance Reduction for Finite-sum Non-convex Problems},
  author = {Dmitry Pasechnyuk-Vilensky and Dmitry Kamzolov and Martin Takáč},
  journal= {arXiv preprint arXiv:2510.08714},
  year   = {2026}
}

Comments

14 pages

R2 v1 2026-07-01T06:27:54.582Z