Related papers: Gradient flow on extensive-rank positive semi-defi…
In this paper, we propose an unfitted finite element method to solve PDE-constrained shape optimization problems via shape gradient flow. The shape gradient flow system consists of the state equation, the adjoint equation, the velocity…
Currently existing energy-stable parametric finite element methods for surface diffusion flow and other flows are usually limited to first-order accuracy in time. Designing a high-order algorithm for geometric flows that can also be…
In this paper, we study randomized and cyclic coordinate descent for convex unconstrained optimization problems. We improve the known convergence rates in some cases by using the numerical semidefinite programming performance estimation…
We propose a positivity preserving entropy decreasing finite volume scheme for nonlinear nonlocal equations with a gradient flow structure. These properties allow for accurate computations of stationary states and long-time asymptotics…
We prove that the standard gradient flow in parameter space that underlies many training algorithms in deep learning can be continuously deformed into an adapted gradient flow which yields (constrained) Euclidean gradient flow in output…
The recipe behind the success of deep learning has been the combination of neural networks and gradient-based optimization. Understanding the behavior of gradient descent however, and particularly its instability, has lagged behind its…
Several key questions remain unanswered regarding overparameterized learning models. It is unclear how (stochastic) gradient descent finds solutions that generalize well, and in particular the role of small random initializations. Matrix…
We study the running of the coupling in SU(2) gauge theory with 8 massless fundamental representation fermion flavours, using the gradient flow method with the Schr\"odinger functional boundary conditions. Gradient flow allows us to measure…
The phase space flow of a dynamical system leading to the solution of Linear Programming (LP) problems is explored as an example of complexity analysis in an analog computation framework. An ensemble of LP problems with $n$ variables and…
When optimizing over-parameterized models, such as deep neural networks, a large set of parameters can achieve zero training error. In such cases, the choice of the optimization algorithm and its respective hyper-parameters introduces…
Resolvent analysis of the linearized Navier-Stokes equations provides useful insight into the dynamics of transitional and turbulent flows and can provide a model for the dominant coherent structures within the flow, particularly for flows…
In this work, we address a foundational question in the theoretical analysis of the Deep Ritz Method (DRM) under the over-parameteriztion regime: Given a target precision level, how can one determine the appropriate number of training…
We present a new flow framework for separation logic reasoning about programs that manipulate general graphs. The framework overcomes problems in earlier developments: it is based on standard fixed point theory, guarantees least flows,…
Gradient-based learning in multi-layer neural networks displays a number of striking features. In particular, the decrease rate of empirical risk is non-monotone even after averaging over large batches. Long plateaus in which one observes…
We introduce and implement a method to compute stationary states of nonlinear Schr\''odinger equations on metric graphs. Stationary states are obtained as local minimizers of the nonlinear Schr\''odinger energy at fixed mass. Our method is…
Multilevel optimization has gained renewed interest in machine learning due to its promise in applications such as hyperparameter tuning and continual learning. However, existing methods struggle with the inherent difficulty of efficiently…
Motivated by the emerging role of interpolating machines in signal processing and machine learning, this work considers the computational aspects of over-parametrized matrix factorization. In this context, the optimization landscape may…
We present a method of parameter estimation for large class of nonlinear systems, namely those in which the state consists of output derivatives and the flow is linear in the parameter. The method, which solves for the unknown parameter by…
We provide a rigorous analysis of implicit regularization in an overparametrized tensor factorization problem beyond the lazy training regime. For matrix factorization problems, this phenomenon has been studied in a number of works. A…
In this paper, we propose a unified convergence analysis for a class of generic shuffling-type gradient methods for solving finite-sum optimization problems. Our analysis works with any sampling without replacement strategy and covers many…