Related papers: Convergence Results for Implicit--Explicit General…
We study the gradient descent (GD) dynamics of a depth-2 linear neural network with a single input and output. We show that GD converges at an explicit linear rate to a global minimum of the training loss, even with a large stepsize --…
Many hyperbolic and kinetic equations contain a non-stiff convection/transport part and a stiff relaxation/collision part (characterized by the relaxation or mean free time $\varepsilon$). To solve this type of problems, implicit-explicit…
In this work we study different Implicit-Explicit (IMEX) schemes for incompressible flow problems with variable viscosity. Unlike most previous work on IMEX schemes, which focuses on the convective part, we here focus on treating parts of…
This paper presents and investigates an inexact proximal gradient method for solving composite convex optimization problems characterized by an objective function composed of a sum of a full-domain differentiable convex function and a…
This article proposes a new class of general linear method with $p=q$ and $r=s=p+1$. The construction of the present method is carried out using order conditions and error minimization subject to $A$- stability constraints. The proposed…
We analyze the convergence of the Conjugate Gradient (CG) method in exact arithmetic, when the coefficient matrix $A$ is symmetric positive semidefinite and the system is consistent. To do so, we diagonalize $A$ and decompose the algorithm…
We show that, even for extremely stiff systems, explicit integration may compete in both accuracy and speed with implicit methods if algebraic methods are used to stabilize the numerical integration. The required stabilizing algebra depends…
We propose a new stochastic gradient method for optimizing the sum of a finite set of smooth functions, where the sum is strongly convex. While standard stochastic gradient methods converge at sublinear rates for this problem, the proposed…
A cell-centered implicit-explicit updated Lagrangian finite volume scheme on unstructured grids is proposed for a unified first order hyperbolic formulation of continuum fluid and solid mechanics. The scheme provably respects the stiff…
Uniformly regular equilibrium problems are natural generalizations of abstract equilibrium prob lems and they are defined over the uniformly prox-regular nonconvex sets. Some new efficient implicit methods for solving uniformly regular…
Many problems in statistics and machine learning can be formulated as model selection problems, where the goal is to choose an optimal parsimonious model among a set of candidate models. It is typical to conduct model selection by…
We consider the development of high order asymptotic-preserving linear multistep methods for kinetic equations and related problems. The methods are first developed for BGK-like kinetic models and then extended to the case of the full…
In this article a unified approach to iterative soft-thresholding algorithms for the solution of linear operator equations in infinite dimensional Hilbert spaces is presented. We formulate the algorithm in the framework of generalized…
Time integration methods for solving initial value problems are an important component of many scientific and engineering simulations. Implicit time integrators are desirable for their stability properties, significantly relaxing…
For solving linear ill-posed problems regularization methods are required when the right hand side is with some noise. In the present paper regularized solutions are obtained by implicit iteration methods in Hilbert scales. % By exploiting…
Gradient schemes is a framework that enables the unified convergence analysis of many numerical methods for elliptic and parabolic partial differential equations: conforming and non-conforming Finite Element, Mixed Finite Element and Finite…
Efficient long-time integration of nonlinear fractional differential equations is significantly challenging due to the integro-differential nature of the fractional operators. In addition, the inherent non-smoothness introduced by the…
Implicit-Explicit (IMEX) methods are flexible numerical time integration methods which solve an initial-value problem (IVP) that is partitioned into stiff and nonstiff processes with the goal of lower computational costs than a purely…
We present consistent algorithms for multiclass learning with complex performance metrics and constraints, where the objective and constraints are defined by arbitrary functions of the confusion matrix. This setting includes many common…
This work establishes new convergence guarantees for gradient descent in smooth convex optimization via a computer-assisted analysis technique. Our theory allows nonconstant stepsize policies with frequent long steps potentially violating…