Related papers: Gradient Flows and Nonlinear Power Methods for the…
Variational interpolants are an indispensable tool for the construction of gradient-flow solutions via the Minimizing Movement Scheme. The De Giorgi lemma provides the associated discrete energy-dissipation inequality. It was originally…
We study the implicit regularization imposed by gradient descent for learning multi-layer homogeneous functions including feed-forward fully connected and convolutional deep neural networks with linear, ReLU or Leaky ReLU activation. We…
In this article, the existence of the spectrum (the eigenvalues) for the nonlinear continuous operators acting in the Banach spaces is investigated. For the study, this question is used a different approach that allows the studying of all…
We study non-convex subgradient flows for training two-layer ReLU neural networks from a convex geometry and duality perspective. We characterize the implicit bias of unregularized non-convex gradient flow as convex regularization of an…
In this paper, we propose novel algorithms integrated projection-free techniques with accelerated gradient flows to minimize bending energies for nonlinear plates with non-convex metric constraints. We discuss the stability and constraint…
We derive explicit equations governing the cumulative biases and weights in Deep Learning with ReLU activation function, based on gradient descent for the Euclidean cost in the input layer, and under the assumption that the weights are, in…
In proper, geodesic Gromov hyperbolic spaces, we investigate discrete-time gradient flows via the proximal point algorithm for unbounded Lipschitz convex functions. Assuming that the target convex function has negative asymptotic slope…
This paper presents and investigates an inexact proximal gradient method for solving composite convex optimization problems characterized by an objective function composed of a sum of a full-domain differentiable convex function and a…
In this paper, we study the asymptotic behavior of continuous- and discrete-time gradient flows of a ``lower-unbounded" convex function $f$ on a Hadamard manifold $M$, particularly, their convergence properties to the boundary $M^{\infty}$…
We introduce a new framework to deal with rough differential equations based on flows and their approximations. Our main result is to prove that measurable flows exist under weak conditions, even solutions to the corresponding rough…
We explicitly construct parameter transformations between gradient flows in metric spaces, called curves of maximal slope, having different exponents when the associated function satisfies a suitable convexity condition. These…
In this work, we develop an $\mathcal{O}(N)$ implicit real space method in 1D and 2D for the Cahn Hilliard (CH) and vector Cahn Hilliard (VCH) equations, based on the Method Of Lines Transpose (MOL$^\text{T}$) formulation. This formulation…
Convolutional neural networks are widely used in imaging and image recognition. Learning such networks from training data leads to the minimization of a non-convex function. This makes the analysis of standard optimization methods such as…
Composite optimization problems, where the sum of a smooth and a merely lower semicontinuous function has to be minimized, are often tackled numerically by means of proximal gradient methods as soon as the lower semicontinuous part of the…
We consider the proximal-gradient method for minimizing an objective function that is the sum of a smooth function and a non-smooth convex function. A feature that distinguishes our work from most in the literature is that we assume that…
We prove an implicit function theorem for functions on infinite-dimensional Banach manifolds, invariant under the (local) action of a finite dimensional Lie group. Motivated by some geometric variational problems, we consider group actions…
In this work, we present a new approach to analyze the gradient flow for a positive semi-definite matrix denoising problem in an extensive-rank and high-dimensional regime. We use recent linear pencil techniques of random matrix theory to…
This article overviews how gradient flows, and discretizations thereof, are useful to design and analyze optimization and sampling algorithms. The interplay between optimization, sampling, and gradient flows is an active research area; our…
We present a framework to calculate large deviations for nonlinear functions of independent random variables supported on compact sets in Banach spaces, by extending the result in Chatterjee and Dembo [6]. Previous research on nonlinear…
We investigate the renormalization group (RG) structure of the gradient flow. Instead of using the original bare action to generate the flow, we propose to use the effective action at each flow time. We write down the basic equation for…