Related papers: Continuous-Time Analysis for Minimax and Bilevel P…
In recent years, bilevel approaches have become very popular to efficiently estimate high-dimensional hyperparameters of machine learning models. However, to date, binary parameters are handled by continuous relaxation and rounding…
This paper focuses on stochastic methods for solving smooth non-convex strongly-concave min-max problems, which have received increasing attention due to their potential applications in deep learning (e.g., deep AUC maximization,…
We examine a class of stochastic differential inclusions involving multiscale effects designed to solve a class of generalized variational inequalities. This class of problems contains constrained convex non-smooth optimization problems,…
In this paper, we study a bilinear saddle point problem of the form $\min_{x}\max_{y} F(x) + \langle Ax, y \rangle - G(y)$, where $F$ and $G$ are $\mu_F$- and $\mu_G$-strongly convex functions, respectively. By incorporating Nesterov…
We investigate a family of approximate multi-step proximal point methods, framed as implicit linear discretizations of gradient flow. The resulting methods are multi-step proximal point methods, with similar computational cost in each…
Stability analysis and control of linear impulsive systems is addressed in a hybrid framework, through the use of continuous-time time-varying discontinuous Lyapunov functions. Necessary and sufficient conditions for stability of impulsive…
Stochastic gradient methods with momentum are widely used in applications and at the core of optimization subroutines in many popular machine learning libraries. However, their sample complexities have not been obtained for problems beyond…
This study develops a framework for a class of constant modulus (CM) optimization problems, which covers binary constraints, discrete phase constraints, semi-orthogonal matrix constraints, non-negative semi-orthogonal matrix constraints,…
We extend slow manifolds near a transcritical singularity in a fast-slow system given by the explicit Euler discretization of the corresponding continuous-time normal form. The analysis uses the blow-up method and direct trajectory-based…
Accelerated gradient descent iterations are widely used in optimization. It is known that, in the continuous-time limit, these iterations converge to a second-order differential equation which we refer to as the accelerated gradient flow.…
Sparse inversion and classification problems are ubiquitous in modern data science and imaging. They are often formulated as non-smooth minimisation problems. In sparse inversion, we minimise, e.g., the sum of a data fidelity term and an…
In this work we present a theoretical and numerical study of the behaviour of the maximum Lyapunov exponent for a generic coupled-map-lattice in the weak-coupling regime. We explain the observed results by introducing a suitable…
We introduce the "continuized" Nesterov acceleration, a close variant of Nesterov acceleration whose variables are indexed by a continuous time parameter. The two variables continuously mix following a linear ordinary differential equation…
In this work, we study first-order algorithms for solving Bilevel Optimization (BO) where the objective functions are smooth but possibly nonconvex in both levels and the variables are restricted to closed convex sets. As a first step, we…
This paper concerns a new class of discontinuous dynamical systems for constrained optimization. These dynamics are particularly suited to solve nonlinear, non-convex problems in closed-loop with a physical system. Such approaches using…
This paper considers time-average optimization, where a decision vector is chosen every time step within a (possibly non-convex) set, and the goal is to minimize a convex function of the time averages subject to convex constraints on these…
We provide Lyapunov-like characterizations of boundedness and convergence of non-trivial solutions for a class of systems with unstable invariant sets. Examples of systems to which the results may apply include interconnections of stable…
Bilevel optimization problems embed the optimality of a subproblem as a constraint of another optimization problem. We introduce the concept of near-optimality robustness for bilevel optimization, protecting the upper-level solution…
For minimizing a strongly convex objective function subject to linear inequality constraints, we consider a penalty approach that allows one to utilize stochastic methods for problems with a large number of constraints and/or objective…
In a Hilbert space, we propose a class of general mixed-order primal-dual dynamical systems with Tikhonov regularization for a convex optimization problem with linear equality constraints. The proposed dynamical system is characterized by…