Related papers: Gradient extremals, talwegs, valleys, and directio…
For addressing optimisation tasks on finite dimensional quantum systems, we give a comprehensive account of the foundations of gradient flows on Riemannian manifolds including new developments: we extend former results from Lie groups such…
We study evolution equations on metric graphs with reservoirs, that is graphs where a one-dimensional interval is associated to each edge and, in addition, the vertices are able to store and exchange mass with these intervals. Focusing on…
Recent studies of gradient descent with large step sizes have shown that there is often a regime with an initial increase in the largest eigenvalue of the loss Hessian (progressive sharpening), followed by a stabilization of the eigenvalue…
The performance of optimization methods is often tied to the spectrum of the objective Hessian. Yet, conventional assumptions, such as smoothness, do often not enable us to make finely-grained convergence statements -- particularly not for…
Let E be the Engel group and D be a rank 2 bracket generating left invariant distribution with a Lorentzian metric, which is a nondegenerate metric of index 1. In this paper, we first prove that timelike normal extremals are locally…
Classical gradient systems have a linear relation between rates and driving forces. In generalized gradient systems we allow for arbitrary relations derived from general non-quadratic dissipation potentials. This paper describes two natural…
For a map that is strictly but not strongly convex, model-based gradient extremum seeking has an eigenvalue of zero at the extremum, i.e., it fails at exponential convergence. Interestingly, perturbation-based model-free extremum seeking…
Generalized Berwald manifolds are Finsler manifolds admitting linear connections such that the parallel transports preserve the Finslerian length of tangent vectors (compatibi\-li\-ty condition). By the fundamental result of the theory…
We present a new class of gradient-type optimization methods that extends vanilla gradient descent, mirror descent, Riemannian gradient descent, and natural gradient descent. Our approach involves constructing a surrogate for the objective…
It seems that in the current age, computers, computation, and data have an increasingly important role to play in scientific research and discovery. This is reflected in part by the rise of machine learning and artificial intelligence,…
We derive methods to compute higher order differentials (Hessians and Hessian-vector products) of the rendering operator. Our approach is based on importance sampling of a convolution that represents the differentials of rendering…
This paper is devoted to the investigation of gradient flows in asymmetric metric spaces (for example, irreversible Finsler manifolds and Minkowski normed spaces) by means of discrete approximation. We study basic properties of curves and…
We give curvature-dependant convergence rates for the optimization of weakly convex functions defined on a manifold of 1-bounded geometry via Riemannian gradient descent and via the dynamic trivialization algorithm. In order to do this, we…
We decompose the velocity gradient tensor for turbulence into normal and non-normal parts, and condition our analysis on the strain eigenvector alignments between these tensors. We identify states that always enhance, and always counteract…
Sign-based optimization methods have become popular in machine learning due to their favorable communication cost in distributed optimization and their surprisingly good performance in neural network training. Furthermore, they are closely…
In a real Hilbert space setting, we study the convergence properties of an inexact gradient algorithm featuring both viscous and Hessian driven damping for convex differentiable optimization. In this algorithm, the gradient evaluation can…
Analyzing the fluid velocity gradients in a Lagrangian reference frame provides an insightful way to study the small-scale dynamics of turbulent flows, and further insight is provided by considering the equations in the eigenframe of the…
The Edge of Stability (EoS) is a phenomenon where the sharpness (largest eigenvalue) of the Hessian approaches and then hovers near the stability threshold $2/\eta$ during gradient descent (GD) with step size $\eta$. Despite (apparently)…
This paper surveys some recent results and progress on the extremal prob- lems in a given set consisting of all simple connected graphs with the same graphic degree sequence. In particular, we study and characterize the extremal graphs…
Existing analyses of optimization in deep learning are either continuous, focusing on (variants of) gradient flow, or discrete, directly treating (variants of) gradient descent. Gradient flow is amenable to theoretical analysis, but is…