Related papers: On semi-convergence of generalized skew-Hermitian …
Minimizing sum of two functions under a linear constraint is what we called splitting problem. This convex optimization has wide applications in machine learning problems, such as Lasso, Group Lasso and Sparse logistic regression. A recent…
We analyze stochastic gradient descent for optimizing non-convex functions. In many cases for non-convex functions the goal is to find a reasonable local minimum, and the main concern is that gradient updates are trapped in saddle points.…
The indefinite least squares (ILS) problem is a generalization of the famous linear least squares problem. It minimizes an indefinite quadratic form with respect to a signature matrix. For this problem, we first propose an impressively…
Over the fast few years, the numerical success of the generalized alternating direction method of multipliers (GADMM) proposed by Eckstein \& Bertsekas [Math. Prog., 1992] has inspired intensive attention in analyzing its theoretical…
Methods with adaptive scaling of different features play a key role in solving saddle point problems, primarily due to Adam's popularity for solving adversarial machine learning problems, including GANS training. This paper carries out a…
In this paper we examine iterative methods for solving the forward ($A{\bf x}={\bf b}$) and adjoint ($A^{T}{\bf y}={\bf g}$) systems of linear equations used to approximate the scattering amplitude, defined by ${\bf g}^{T}{\bf x}={\bf…
In this paper, we extend the inexact Uzawa algorithm in [Q. Hu, J. Zou, SIAM J. Matrix Anal., 23(2001), pp. 317-338] to the nonsymmetric generalized saddle point problem. The techniques used here are similar to those in [Bramble \emph{et…
This paper proposes a parametric error analysis method for Goldschmidt floating point division, which reveals how the errors of the intermediate results accumulate and propagate during the Goldschmidt iterations. The analysis is developed…
The present work provides a comprehensive study of symmetric-conjugate operator splitting methods in the context of linear parabolic problems and demonstrates their additional benefits compared to symmetric splitting methods. Relevant…
Machine learning solvers for partial differential equations (PDEs) have attracted growing interest. However, most existing approaches, such as neural network solvers, rely on stochastic training, which is inefficient and typically requires…
We propose an inexact Uzawa algorithm with two variable relaxation parameters for solving the generalized saddle-point system. The saddle-point problems can be found in a wide class of applications, such as the augmented Lagrangian…
We present an abstract concept for the error analysis of numerical schemes for semilinear stochastic partial differential equations (SPDEs) and demonstrate its usefulness by proving the strong convergence of a Milstein-Galerkin finite…
Design optimization and uncertainty quantification, among other applications of industrial interest, require fast or multiple queries of some parametric model. The Proper Generalized Decomposition (PGD) provides a separable solution, a…
Recently, saddle point problems have received much attention due to their powerful modeling capability for a lot of problems from diverse domains. Applications of these problems occur in many applied areas, such as robust optimization,…
In this work, we study the asymptotic randomness of an algorithmic estimator of the saddle point of a globally convex-concave and locally strongly-convex strongly-concave objective. Specifically, we show that the averaged iterates of a…
We present a scalable iterative solver for high-order hybridized discontinuous Galerkin (HDG) discretizations of linear partial differential equations. It is an interplay between domain decomposition methods and HDG discretizations, and…
We consider a two dimensional biharmonic problem and its discretization by means of a symmetric interior penalty discontinuous Galerkin method. A novel split of an error measure based on a generalized Hessian into two terms measuring the…
In this paper, we introduce a multilevel algorithm for approximating variational formulations of symmetric saddle point systems. The algorithm is based on availability of families of stable finite element pairs and on the availability of…
Non-convex sparsity-inducing penalties have recently received considerable attentions in sparse learning. Recent theoretical investigations have demonstrated their superiority over the convex counterparts in several sparse learning…
In this article we investigate a finite element formulation of strongly monotone quasi-linear elliptic PDEs in the context of fixed-point iterations. As opposed to Newton's method, which requires information from the previous iteration in…