带可证明调优学习率方案的梯度下降
核实验
2026-02-06 v3
摘要
梯度基迭代优化方法是现代机器学习的核心工具，关键依赖于学习率等参数的精细调优。然而，这些参数通常采用启发式方法设置，缺乏形式的近最优保证。最近的工作由Gupta和Roughgarden研究如何学习梯度下降中的优秪步长。然而，与大多数具有理论保证的梯度优化文献一样，其结果依赖于函数类的强假设，包括凸性和光滑性，这些假设在典型应用中难以满足。本文我们发展出新的分析工具，用于在非凸、非光滑函数上对梯度算法中的超参数进行可证明的调优。我们获得的样本复杂度界与先前在光滑、凸函数上的结果相符（在对数因子上），但适用于更广泛的函数类。我们的分析适用于常用激活函数（包括ReLU、sigmoid和tanh）的神经网络梯度下降。我们将框架扩展到同时调优多个超参数，包括学习率方案、动量及初始化向量。我们的方法可用于界定最小化验证损失以及梯度下降迭代次数的样本复杂度。
引用
@article{arxiv.2512.05083,
title = {First Study of the Nuclear Response to Fast Hadrons via Angular Correlations between Pions and Slow Protons in Electron-Nucleus Scattering},
author = {S. J. Paul and M. Arratia and H. Hakobyan and W. Brooks and A. Acar and P. Achenbach and J. S. Alvarado and W. R. Armstrong and N. A. Baltzell and L. Barion and M. Bashkanov and M. Battaglieri and F. Benmokhtar and A. Bianconi and A. S. Biselli and F. Bossù and S. Boiarinov and K. -T. Brinkmann and W. J. Briscoe and V. Burkert and T. Cao and D. S. Carman and P. Chatagnon and H. Chinchay and G. Ciullo and P. L. Cole and A. D'Angelo and N. Dashyan and R. De Vita and A. Deur and S. Diehl and C. Djalali and R. Dupre and H. Egiyan and A. El Alaoui and L. Elouadrhiri and P. Eugenio and M. Farooq and S. Fegan and A. Filippi and C. Fogler and G. Gavalian and G. P. Gilfoyle and R. W. Gothe and B. Gualtieri and M. Hattawy and F. Hauenstein and T. B. Hayward and M. Hoballah and M. Holtrop and Yu-Chun Hung and Y. Ilieva and D. G. Ireland and E. L. Isupov and D. Jenkins and H. S. Jo and D. Keller and M. Khandaker and A. Kim and V. Klimenko and I. Korover and A. Kripko and V. Kubarovsky and L. Lanza and S. Lee and P. Lenisa and X. Li and D. Marchand and V. Mascagna and B. McKinnon and T. Mineeva and V. Mokeev and E. F. Molina Cardenas and C. Munoz Camacho and P. Nadel-Turonski and T. Nagorna and K. Neupane and S. Niccolai and G. Niculescu and M. Osipenko and A. I. Ostrovidov and M. Ouillon and P. Pandey and M. Paolone and L. L. Pappalardo and R. Paremuzyan and E. Pasyuk and C. Paudel and W. Phelps and N. Pilleux and P. S. H. Vaishnavi and S. Polcher Rafael and L. Polizzi and J. W. Price and Y. Prok and A. Radic and T. Reed and J. Richards and M. Ripani and J. Ritman and G. Rosner and S. Schadmand and A. Schmidt and R. A. Schumacher and Y. Sharabian and S. Shrestha and E. Sidoretti and D. Sokhan and N. Sparveris and M. Spreafico and S. Stepanyan and I. I. Strakovsky and S. Strauch and M. Tenorio and F. Touchte Codjo and R. Tyson and M. Ungaro and S. Vallarino and C. Velasquez and L. Venturelli and H. Voskanyan and E. Voutier and Y. Wang and D. P. Watts and U. Weerasinghe and X. Wei and M. H. Wood and L. Xu and Z. Xu and M. Zurek},
journal= {arXiv preprint arXiv:2512.05083},
year = {2026}
}