An Adaptively Inexact Method for Bilevel Learning Using Primal-Dual Style Differentiation
Abstract
We consider a bilevel learning framework for learning linear operators. In this framework, the learnable parameters are optimized via a loss function that also depends on the minimizer of a convex optimization problem (denoted lower-level problem). We utilize an iterative algorithm called `piggyback' to compute the gradient of the loss and minimizer of the lower-level problem. Given that the lower-level problem is solved numerically, the loss function and thus its gradient can only be computed inexactly. To estimate the accuracy of the computed hypergradient, we derive an a-posteriori error bound, which provides guides for setting the tolerance for the lower-level problem, as well as the piggyback algorithm. To efficiently solve the upper-level optimization, we also propose an adaptive method for choosing a suitable step-size. To illustrate the proposed method, we consider a few learned regularizer problems, such as training an input-convex neural network.
Cite
@article{arxiv.2412.06436,
title = {An Adaptively Inexact Method for Bilevel Learning Using Primal-Dual Style Differentiation},
author = {Lea Bogensperger and Matthias J. Ehrhardt and Thomas Pock and Mohammad Sadegh Salehi and Hok Shing Wong},
journal= {arXiv preprint arXiv:2412.06436},
year = {2025}
}