Related papers: Nonlinearly preconditioned gradient flows
In this work, we approach the minimization of a continuously differentiable convex function under linear equality constraints by a second-order dynamical system with asymptotically vanishing damping term. The system is formulated in terms…
This paper introduces a novel closed-form strategy that dynamically modifies the reference of a pre-compensated nonlinear system to ensure the satisfaction of a set of convex constraints. The main idea consists of translating constraints in…
This article is devoted to presenting an abstract theory on time-fractional gradient flows for nonconvex energy functionals in Hilbert spaces. Main results consist of local and global in time existence of (continuous) strong solutions to…
Quadratic Lyapunov functions are prevalent in stability analysis of linear consensus systems. In this paper we show that weighted sums of convex functions of the different coordinates are Lyapunov functions for irreducible consensus…
Convolutional neural networks are widely used in imaging and image recognition. Learning such networks from training data leads to the minimization of a non-convex function. This makes the analysis of standard optimization methods such as…
We present a new algorithm for solving optimization problems with objective functions that are the sum of a smooth function and a (potentially) nonsmooth regularization function, and nonlinear equality constraints. The algorithm may be…
We investigate the gradient flow of the $L^2$ norm of the Riemannian curvature on surfaces. We show long time existence with arbitrary initial data, and exponential convergence of the volume normalized flow to a constant scalar curvature…
We provide an overview of primal-dual algorithms for nonsmooth and non-convex-concave saddle-point problems. This flows around a new analysis of such methods, using Bregman divergences to formulate simplified conditions for convergence.
We consider (stochastic) subgradient methods for strongly convex but potentially nonsmooth non-Lipschitz optimization. We provide new equivalent dual descriptions (in the style of dual averaging) for the classic subgradient method, the…
We study a Lagrangian numerical scheme for solution of a nonlinear drift diffusion equation of the form $\partial_t u = \partial_x(u \cdot c[\partial_x(h^\prime(u)+v)])$ on an interval. This scheme will consist of a spatio-temporal…
The Lagrangian derivatives of finite-time Lyapunov exponents and the corresponding characteristic directions are shown to satisfy time-asymptotic differential constraints in chaotic flows. The constraints are valid for any metric tensor,…
Preconditioning is a crucial operation in gradient-based numerical optimisation. It helps decrease the local condition number of a function by appropriately transforming its gradient. For a convex function, where the gradient can be…
Motivated by an inertial primal-dual dynamical system with vanishing damping, we propose a class of accelerated augmented Lagrangian methods with Nesterov extrapolation parameters for a linearly constrained convex optimization problem with…
The mirror descent algorithm is known to be effective in situations where it is beneficial to adapt the mirror map to the underlying geometry of the optimization model. However, the effect of mirror maps on the geometry of distributed…
In this paper, we consider a nonsmooth convex finite-sum problem with a conic constraint. To overcome the challenge of projecting onto the constraint set and computing the full (sub)gradient, we introduce a primal-dual incremental gradient…
The paper surveys recent progresses in understanding the dynamics and loss landscape of the gradient flow equations associated to deep linear neural networks, i.e., the gradient descent training dynamics (in the limit when the step size…
We develop a unified theory of augmented Lagrangians for nonconvex optimization problems that encompasses both duality theory and convergence analysis of primal-dual augmented Lagrangian methods in the infinite dimensional setting. Our goal…
In this paper, we study the implicit regularization of the gradient descent algorithm in homogeneous neural networks, including fully-connected and convolutional neural networks with ReLU or LeakyReLU activations. In particular, we study…
In this paper, we propose a descent method for composite optimization problems with linear operators. Specifically, we first design a structure-exploiting preconditioner tailored to the linear operator so that the resulting preconditioned…
The success of deep learning over the past decade mainly relies on gradient-based optimisation and backpropagation. This paper focuses on analysing the performance of first-order gradient-based optimisation algorithms, gradient descent and…