Related papers: Nonlinearly preconditioned gradient flows
Recent results have shown that for two-layer fully connected neural networks, gradient flow converges to a global optimum in the infinite width limit, by making a connection between the mean field dynamics and the Wasserstein gradient flow.…
Converse optimality theory addresses an optimal control problem conversely where the system is unknown and the value function is chosen. Previous work treated this problem both in continuous and discrete time and non-extensively considered…
We deal with the convergence of the value function of an approximate control problem with uncertain dynamics to the value function of a nonlinear optimal control problem. The assumptions on the dynamics and the costs are rather general and…
We prove the well-posedness of entropy solutions for a wide class of nonlocal transport equations with nonlinear mobility in one spatial dimension. The solution is obtained as the limit of approximations constructed via a deterministic…
We investigate a family of approximate multi-step proximal point methods, accelerated by implicit linear discretizations of gradient flow. The resulting methods are multi-step proximal point methods, with similar computational cost in each…
We investigate a variational approach to nonpotential perturbations of gradient flows of nonconvex energies in Hilbert spaces. We prove existence of solutions to elliptic-in-time regularizations of gradient flows by combining the…
We propose a new normalized Sobolev gradient flow for the Gross-Pitaevskii eigenvalue problem based on an energy inner product that depends on time through the density of the flow itself. The gradient flow is well-defined and converges to…
We analyze the global and local behavior of gradient-like flows under stochastic errors towards the aim of solving convex optimization problems with noisy gradient input. We first study the unconstrained differentiable convex case, using a…
Foundations of a new projection-based model reduction approach for convection dominated nonlinear fluid flows are summarized. In this method the evolution of the flow is approximated in the Lagrangian frame of reference. Global basis…
Accelerated gradient methods are the cornerstones of large-scale, data-driven optimization problems that arise naturally in machine learning and other fields concerning data analysis. We introduce a gradient-based optimization framework for…
The generalization mystery of overparametrized deep nets has motivated efforts to understand how gradient descent (GD) converges to low-loss solutions that generalize well. Real-life neural networks are initialized from small random values…
Composite optimization problems, where the sum of a smooth and a merely lower semicontinuous function has to be minimized, are often tackled numerically by means of proximal gradient methods as soon as the lower semicontinuous part of the…
In this note, we prove that the abstract gradient flow introduced by Baird-Fardoun-Regbaoui \cite{BFR}is well-posed on a closed Riemann surface with conical singularity. Long time existence and convergence of the flow are proved under…
We introduce action-driven flows for causal variational principles, being a class of non-convex variational problems emanating from applications in fundamental physics. In the compact setting, H\"older continuous curves of measures are…
We study the implicit bias of gradient descent methods in solving a binary classification problem over a linearly separable dataset. The classifier is described by a nonlinear ReLU model and the objective function adopts the exponential…
Recently, there has been great interest in connections between continuous-time dynamical systems and optimization methods, notably in the context of accelerated methods for smooth and unconstrained problems. In this paper we extend this…
The initial-value problem for the perturbed gradient flow \[ B(t,u(t)) \in \partial\Psi_{u(t)}(u'(t))+\partial \mathcal E_t(u(t)) \text{ for a.a. } t\in (0,T),\qquad u(0)=u_0 \] with a perturbation $B$ in a Banach space $V$ is investigated,…
We show that a wide class of risk-constrained nonconvex functional optimization problems exhibit strong duality, regardless of nonconvexity. We develop two novel results under distinct sets of assumptions, establishing strong duality over…
In this work we study the convergence of gradient methods for nonconvex optimization problems -- specifically the effect of the problem formulation to the convergence behavior of the solution of a gradient flow. We show through a simple…
Motivated by energy based analyses for descent methods in the Euclidean setting, we investigate a generalisation of such analyses for descent methods over Riemannian manifolds. In doing so, we find that it is possible to derive…