"Active-set complexity" of proximal gradient: How long does it take to find the sparsity pattern?
Abstract
Proximal gradient methods have been found to be highly effective for solving minimization problems with non-negative constraints or L1-regularization. Under suitable nondegeneracy conditions, it is known that these algorithms identify the optimal sparsity pattern for these types of problems in a finite number of iterations. However, it is not known how many iterations this may take. We introduce the notion of the "active-set complexity", which in these cases is the number of iterations before an algorithm is guaranteed to have identified the final sparsity pattern. We further give a bound on the active-set complexity of proximal gradient methods in the common case of minimizing the sum of a strongly-convex smooth function and a separable convex non-smooth function.
Cite
@article{arxiv.1712.03577,
title = {"Active-set complexity" of proximal gradient: How long does it take to find the sparsity pattern?},
author = {Julie Nutini and Mark Schmidt and Warren Hare},
journal= {arXiv preprint arXiv:1712.03577},
year = {2018}
}
Comments
Added discussion of Liang et al. (2017), which previously analyzed active-set complexity of the proximal-gradient (PG) method in the convex setting under an additional assumption. Added convergence analysis of the PG method under a general constant step-size for strongly-convex functions, use this to bound the active-set complexity under a general constant step-size. Additional minor edits