Generalization Error of $f$-Divergence Stabilized Algorithms via Duality
Machine Learning
2025-02-21 v1 Machine Learning
Abstract
The solution to empirical risk minimization with -divergence regularization (ERM-DR) is extended to constrained optimization problems, establishing conditions for equivalence between the solution and constraints. A dual formulation of ERM-DR is introduced, providing a computationally efficient method to derive the normalization function of the ERM-DR solution. This dual approach leverages the Legendre-Fenchel transform and the implicit function theorem, enabling explicit characterizations of the generalization error for general algorithms under mild conditions, and another for ERM-DR solutions.
Keywords
Cite
@article{arxiv.2502.14544,
title = {Generalization Error of $f$-Divergence Stabilized Algorithms via Duality},
author = {Francisco Daunas and Iñaki Esnaola and Samir M. Perlaza and Gholamali Aminian},
journal= {arXiv preprint arXiv:2502.14544},
year = {2025}
}
Comments
This is new work for ISIT2025. arXiv admin note: text overlap with arXiv:2402.00501