Related papers: A sharp bound on the convergence rate of an aggreg…
In this paper, we propose new linearly convergent second-order methods for minimizing convex quartic polynomials. This framework is applied for designing optimization schemes, which can solve general convex problems satisfying a new…
Due to the high communication cost in distributed and federated learning problems, methods relying on compression of communicated messages are becoming increasingly popular. While in other contexts the best performing gradient-type methods…
We analyze a divergence based first order system least squares method applied to a second order elliptic model problem with homogeneous boundary conditions. We prove optimal convergence in the $L^2(\Omega)$ norm for the scalar variable.…
We study optimal convergence rates in the periodic homogenization of linear elliptic equations of the form $-A(x/\varepsilon):D^2 u^{\varepsilon} = f$ subject to a homogeneous Dirichlet boundary condition. We show that the optimal rate for…
Two-grid methods with exact solution of the Galerkin coarse-grid system have been well studied by the multigrid community: an elegant identity has been established to characterize the convergence factor of exact two-grid methods. In…
We illustrate an efficient new method for handling polynomial systems with degenerate solution sets. In particular, a corollary of our techniques is a new algorithm to find an isolated point in every excess component of the zero set (over…
We consider the decentralized optimization problem, where a network of $n$ agents aims to collaboratively minimize the average of their individual smooth and convex objective functions through peer-to-peer communication in a directed graph.…
Solving structured systems of linear equations in a non-centralized fashion is an important step in many distributed optimization and control algorithms. Fast convergence is required in manifold applications. Known decentralized algorithms,…
An iterative method LSMR is presented for solving linear systems $Ax=b$ and least-squares problem $\min \norm{Ax-b}_2$, with $A$ being sparse or a fast linear operator. LSMR is based on the Golub-Kahan bidiagonalization process. It is…
In this paper we consider from two different aspects the proximal alternating direction method of multipliers (ADMM) in Hilbert spaces. We first consider the application of the proximal ADMM to solve well-posed linearly constrained…
In this paper, we consider the dual formulation of minimizing $\sum_{i\in I}f_i(x_i)+\sum_{j\in J} g_j(\mathcal{A}_jx)$ with the index sets $I$ and $J$ being large. To address the difficulties from the high dimension of the variable $x$…
The quest for an algorithm that solves an $n\times n$ linear system in $O(n^2)$ time complexity, or $O(n^2 \text{poly}(1/\epsilon))$ when solving up to $\epsilon$ relative error, is a long-standing open problem in numerical linear algebra…
We consider the problem of minimizing the sum of a convex function and a convex function composed with an injective linear mapping. For such problems, subject to a coercivity condition at fixed points of the corresponding Picard iteration,…
Sparsity regularized loss minimization problems play an important role in various fields including machine learning, data mining, and modern statistics. Proximal gradient descent method and coordinate descent method are the most popular…
We propose two basic assumptions, under which the rate of convergence of the augmented Lagrange method for a class of composite optimization problems is estimated. We analyze the rate of local convergence of the augmented Lagrangian method…
This paper is concerned with the fundamental problem of estimating chirp parameters from a mixture of linear chirp signals. Unlike most previous methods, which solve the problem by discretizing the parameter space and then estimating the…
Adaptive gradient methods like AdaGrad are widely used in optimizing neural networks. Yet, existing convergence guarantees for adaptive gradient methods require either convexity or smoothness, and, in the smooth setting, only guarantee…
In this paper, we propose an efficient extrapolation cascadic multigrid (EXCMG) method combined with 25-point difference approximation to solve the three-dimensional biharmonic equation. First, through applying Richardson extrapolation and…
In this paper we motivate, discuss the implementation and present the resulting numerics for a new definition of strength of connection which is based on the notion of algebraic distance. This algebraic distance measure, combined with…
Recent studies have shown that proximal gradient (PG) method and accelerated gradient method (APG) with restarting can enjoy a linear convergence under a weaker condition than strong convexity, namely a quadratic growth condition (QGC).…