Related papers: Adjacency Criterion For Gradient Flow With Multipl…

Accelerated gradient descent iterations are widely used in optimization. It is known that, in the continuous-time limit, these iterations converge to a second-order differential equation which we refer to as the accelerated gradient flow.…

We derive the system of differential equations for the gradient flow characterizing the training process of linear in-context learning in full generality. Next, we explore the geometric structure of the gradient flows in two instances,…

This paper deals with local criteria for the convergence to a global minimiser for gradient flow trajectories and their discretisations. To obtain quantitative estimates on the speed of convergence, we consider variations on the classical…

We investigate a family of approximate multi-step proximal point methods, framed as implicit linear discretizations of gradient flow. The resulting methods are multi-step proximal point methods, with similar computational cost in each…

We investigate a family of approximate multi-step proximal point methods, accelerated by implicit linear discretizations of gradient flow. The resulting methods are multi-step proximal point methods, with similar computational cost in each…

We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don't increase the stepsize too fast and 2) don't overstep the local curvature. No need for functional values, no line search, no…

This paper extends previous work from arxiv:1702.05223, which shows that the main theorem of Morse theory holds for a large class of functions on singular spaces, where the function and the underlying singular space are required to satisfy…

This work focuses on the numerical assessment of the accuracy of an adjoint-based gradient in the perspective of variational data assimilation and parameter identification in glaciology. Using noisy synthetic data, we quantify the ability…

This paper is devoted to the investigation of gradient flows in asymmetric metric spaces (for example, irreversible Finsler manifolds and Minkowski normed spaces) by means of discrete approximation. We study basic properties of curves and…

New criteria for energy stability of multi-step, multi-stage, and mixed schemes are introduced in the context of evolution equations that arise as gradient flow with respect to a metric. These criteria are used to exhibit second and third…

Readability criteria, such as distance or neighborhood preservation, are often used to optimize node-link representations of graphs to enable the comprehension of the underlying data. With few exceptions, graph drawing algorithms typically…

We investigate linear-quadratic dynamical systems with energy preserving quadratic terms. These systems arise for instance as Galerkin systems of incompressible flows. A criterion is presented to ensure long-term boundedness of the system…

Bilevel optimization is a key framework in hierarchical decision-making, where one problem is embedded within the constraints of another. In this work, we propose a control-theoretic approach to solving bilevel optimization problems. Our…

We prove almost sure convergence of the maximum degree in an evolving graph model combining a growing number of local choices with sublinear preferential attachment. At each step in the growth of the graph, a new vertex is introduced. Then…

We explicitly construct parameter transformations between gradient flows in metric spaces, called curves of maximal slope, having different exponents when the associated function satisfies a suitable convexity condition. These…

This paper considers the formulation of the adjoint problem in two dimensions when there are shocks in the flow solution. For typical cost functions, the adjoint variables are continuous at shocks, where they have to obey an internal…

We examine a steepest energy descent flow with obstacle constraint in higher order energy frameworks where the maximum principle is not available. We construct the flow under general assumptions using De Giorgi's minimizing movement scheme.…

We consider policy gradient methods for stochastic optimal control problem in continuous time. In particular, we analyze the gradient flow for the control, viewed as a continuous time limit of the policy gradient method. We prove the global…

We propose an adaptive proximal gradient method for minimizing the sum of two functions, where one is a simple convex function, and the other belongs to one of the three classes: nonconvex smooth, convex nonsmooth, or convex smooth. The key…

Minimax problems of the form $\min_x \max_y \Psi(x,y)$ have attracted increased interest largely due to advances in machine learning, in particular generative adversarial networks. These are typically trained using variants of stochastic…