Related papers: Select-then-differentiate: Solving Bilevel Optimiz…
Variational regularization techniques are dominant in the field of mathematical imaging. A drawback of these techniques is that they are dependent on a number of parameters which have to be set by the user. A by now common strategy to…
The optimistic gradient method is useful in addressing minimax optimization problems. Motivated by the observation that the conventional stochastic version suffers from the need for a large batch size on the order of…
Lower-bound analyses for nonconvex strongly-concave minimax optimization problems have shown that stochastic first-order algorithms require at least $\mathcal{O}(\varepsilon^{-4})$ oracle complexity to find an $\varepsilon$-stationary…
We propose adaptive, line search-free second-order methods with optimal rate of convergence for solving convex-concave min-max problems. By means of an adaptive step size, our algorithms feature a simple update rule that requires solving…
Low-rank optimization problems with sparse simplex constraints involve variables that must satisfy nonnegativity, sparsity, and sum-to-1 conditions, making their optimization particularly challenging due to the interplay between low-rank…
The implicit hitting set (IHS) approach offers a general framework for solving computationally hard combinatorial optimization problems declaratively. IHS iterates between a decision oracle used for extracting sources of inconsistency and…
We establish a well-posedness and error-estimation framework that solves Hamilton-Jacobi equations by minimizing the least-squares residual of monotone finite-difference discretizations. This approach also applies naturally to second-order…
This paper proposes a new hybrid high-order discretization for the biharmonic problem and the corresponding eigenvalue problem. The discrete ansatz space includes degrees of freedom in $n-2$ dimensional submanifolds (e.g., nodal values in…
Differential positivity and K-cooperativity, a special case of differential positivity, extend differential approaches to control to nonlinear systems with multiple equilibria, such as switches or multi-agent consensus. To apply this…
Hierarchical optimization refers to problems with interdependent decision variables and objectives, such as minimax and bilevel formulations. While various algorithms have been proposed, existing methods and analyses lack adaptivity in…
Statistical inference from high-dimensional data with low-dimensional structures has recently attracted lots of attention. In machine learning, deep generative modeling approaches implicitly estimate distributions of complex objects by…
The subgradient method is a classical and foundational approach in non-smooth convex optimization; its simplicity, robustness, and role as a conceptual and algorithmic starting point have made it the backbone of many significant…
A gradient-free deterministic method is developed to solve global optimization problems for Lipschitz continuous functions defined in arbitrary path-wise connected compact sets in Euclidean spaces. The method can be regarded as granular…
In this paper, we study multi-block min-max bilevel optimization problems, where the upper level is non-convex strongly-concave minimax objective and the lower level is a strongly convex objective, and there are multiple blocks of dual…
Inverse problems are key issues in several scientific areas, including signal processing and medical imaging. Data-driven approaches for inverse problems aim for learning model and regularization parameters from observed data samples, and…
This paper aims to find efficient solutions to a multi-objective optimization problem (MP) with convex polynomial data. To this end, a hybrid method, which allows us to transform problem (MP) into a scalar convex polynomial optimization…
Bilevel optimization formulates hierarchical decision-making processes that arise in many real-world applications such as in pricing, network design, and infrastructure defense planning. In this paper, we consider a class of bilevel…
We tackle the general differentiable meta learning problem that is ubiquitous in modern deep learning, including hyperparameter optimization, loss function learning, few-shot learning, invariance learning and more. These problems are often…
(Stochastic) bilevel optimization is a frequently encountered problem in machine learning with a wide range of applications such as meta-learning, hyper-parameter optimization, and reinforcement learning. Most of the existing studies on…
Bilevel learning has gained prominence in machine learning, inverse problems, and imaging applications, including hyperparameter optimization, learning data-adaptive regularizers, and optimizing forward operators. The large-scale nature of…