Related papers: On the stability of optimization algorithms given …
A linear evolving surface partial differential equation is first discretized in space by an arbitrary Lagrangian Eulerian (ALE) evolving surface finite element method, and then in time either by a Runge-Kutta method, or by a backward…
Previous algorithms can solve convex-concave minimax problems $\min_{x \in \mathcal{X}} \max_{y \in \mathcal{Y}} f(x,y)$ with $\mathcal{O}(\epsilon^{-2/3})$ second-order oracle calls using Newton-type methods. This result has been…
In recent years, deep learning has been connected with optimal control as a way to define a notion of a continuous underlying learning problem. In this view, neural networks can be interpreted as a discretization of a parametric Ordinary…
This paper presents a numerical method for variable coefficient elliptic PDEs with mostly smooth solutions on two dimensional domains. The PDE is discretized via a multi-domain spectral collocation method of high local order (order 30 and…
There has been a long history of using ordinary differential equations (ODEs) to understand the dynamics of discrete-time algorithms (DTAs). Surprisingly, there are still two fundamental and unanswered questions: (i) it is unclear how to…
Many large-scale and distributed optimization problems can be brought into a composite form in which the objective function is given by the sum of a smooth term and a nonsmooth regularizer. Such problems can be solved via a proximal…
This is the second part of study on the optimal convergence rate of the explicit Euler discretization in time for the convection-diffusion equations [Appl. Math. Lett. \textbf{131} (2022) 108048] which focuses on high-dimensional…
We study a class of algorithms for solving bilevel optimization problems in both stochastic and deterministic settings when the inner-level objective is strongly convex. Specifically, we consider algorithms based on inexact implicit…
High-order entropy-stable discontinuous Galerkin methods for the compressible Euler and Navier-Stokes equations require the positivity of thermodynamic quantities in order to guarantee their well-posedness. In this work, we introduce a…
We review the stability properties of several discretizations of the Helmholtz equation at large wavenumbers. For a model problem in a polygon, a complete $k$-explicit stability (including $k$-explicit stability of the continuous problem)…
An influential line of recent work has focused on the generalization properties of unregularized gradient-based learning procedures applied to separable linear classification with exponentially-tailed loss functions. The ability of such…
We prove lower bounds for higher-order methods in smooth non-convex finite-sum optimization. Our contribution is threefold: We first show that a deterministic algorithm cannot profit from the finite-sum structure of the objective, and that…
We consider a class of stochastic gradient optimization schemes. Assuming that the objective function is strongly convex, we prove weak error estimates which are uniform in time for the error between the solution of the numerical scheme,…
We present an original study on the numerical stabiliy of explicit schemes solving the incompressible Euler equations on an open domain with slipping boundary conditions. Relying on the skewness property of the non-linear term, we…
Subgradient methods are the natural extension to the non-smooth case of the classical gradient descent for regular convex optimization problems. However, in general, they are characterized by slow convergence rates, and they require…
Gradient normalization and soft clipping are two popular techniques for tackling instability issues and improving convergence of stochastic gradient descent (SGD) with momentum. In this article, we study these types of methods through the…
We study the complexity of optimizing highly smooth convex functions. For a positive integer $p$, we want to find an $\epsilon$-approximate minimum of a convex function $f$, given oracle access to the function and its first $p$ derivatives,…
This paper studies fully discrete finite element approximations to the Navier-Stokes equations using inf-sup stable elements and grad-div stabilization. For the time integration two implicit-explicit second order backward differentiation…
The objective of this paper is to prove the convergence of a linear implicit multi-step numerical method for ordinary differential equations. The algorithm is obtained via Taylor approximations. The convergence is proved following the…
In this set of papers we formulate a stand alone method to derive maximal number of linearizing transformations for nonlinear ordinary differential equations (ODEs) of any order including coupled ones from a knowledge of fewer number of…