Related papers: Introduction to Bilevel Optimization: A perspectiv…
It is well known that bilevel optimization problems are hard to solve both in theory and practice. In this paper, we highlight a further computational difficulty when it comes to solving bilevel problems with continuous but nonconvex lower…
These are the lecture notes that accompanied the course of the same name that I taught at the Eindhoven University of Technology from 2021 to 2023. The course is intended as an introduction to neural networks for mathematics students at the…
Functional bilevel methods estimate a lower-level function and plug it into a hypergradient, but this plug-in gradient can retain first-order bias when the lower-level problem is learned nonparametrically. To remove this bias, we develop a…
We investigate the complexity of bilevel combinatorial optimization with uncertainty in the follower's objective, in a robust optimization approach. We show that the robust counterpart of the bilevel problem under interval uncertainty can…
Variational regularization is commonly used to solve linear inverse problems, and involves augmenting a data fidelity by a regularizer. The regularizer is used to promote a priori information and is weighted by a regularization parameter.…
Hyperparameter optimization has remained a central topic within the machine learning community due to its ability to produce state-of-the-art results. With the recent interest growing in the usage of CNNs for time series prediction, we…
Interpretable Multi-Task Learning can be expressed as learning a sparse graph of the task relationship based on the prediction performance of the learned models. Since many natural phenomenon exhibit sparse structures, enforcing sparsity on…
Inverse problem or parameter estimation of ordinary differential equations (ODEs), the iterative process of minimizing the mismatch between model-predicted and experimental states by tuning the parameter values within an optimization…
We propose a new approach to solving bilevel optimization problems, intermediate between solving full-system optimality conditions with a Newton-type approach, and treating the inner problem as an implicit function. The overall idea is to…
With the development of large-scale models, traditional distributed bilevel optimization algorithms cannot be applied directly in low-resource clients. The key reason lies in the excessive computation involved in optimizing both the lower-…
Probabilistic models such as logistic regression, Bayesian classification, neural networks, and models for natural language processing, are increasingly more present in both undergraduate and graduate statistics and data science curricula…
This paper is concerned with the derivation of first- and second-order sufficient optimality conditions for optimistic bilevel optimization problems involving smooth functions. First-order sufficient optimality conditions are obtained by…
A cost functional involving the eigenvalues of an elastic structure, that is described by a multi-phase-field equation, is optimized. This allows us to handle topology changes and multiple materials. We prove continuity and…
Evolutionary reinforcement learning (ERL) algorithms recently raise attention in tackling complex reinforcement learning (RL) problems due to high parallelism, while they are prone to insufficient exploration or model collapse without…
The optimal control of two-level systems by time-dependent laser fields is studied using a variational theory. We obtain, for the first time, general analytical expressions for the optimal pulse shapes leading to global maximization or…
We analyse the asymptotic properties of a continuous-time, two-timescale stochastic approximation algorithm designed for stochastic bilevel optimisation problems in continuous-time models. We obtain the weak convergence rate of this…
Reinforcement learning (RL) has shown significant promise for sequential portfolio optimization tasks, such as stock trading, where the objective is to maximize cumulative returns while minimizing risks using historical data. However,…
The multi-level method for discrete state systems, first introduced by Anderson and Higham [Multiscale Model. Simul. 10:146--179, 2012], is a highly efficient simulation technique that can be used to elucidate statistical characteristics of…
In this paper, we study multi-block min-max bilevel optimization problems, where the upper level is non-convex strongly-concave minimax objective and the lower level is a strongly convex objective, and there are multiple blocks of dual…
This tutorial gives a quick introduction to Variational Bayes (VB), also called Variational Inference or Variational Approximation, from a practical point of view. The paper covers a range of commonly used VB methods and an attempt is made…