On the connections between algorithmic regularization and penalization for convex losses
Optimization and Control
2019-09-10 v1 Machine Learning
Abstract
In this work we establish the equivalence of algorithmic regularization and explicit convex penalization for generic convex losses. We introduce a geometric condition for the optimization path of a convex function, and show that if such a condition is satisfied, the optimization path of an iterative algorithm on the unregularized optimization problem can be represented as the solution path of a corresponding penalized problem.
Cite
@article{arxiv.1909.03371,
title = {On the connections between algorithmic regularization and penalization for convex losses},
author = {Qian Qian and Xiaoyuan Qian},
journal= {arXiv preprint arXiv:1909.03371},
year = {2019}
}