Related papers: Norm-based convergence bounds for nonsymmetric alg…
We introduce a near-linear complexity (geometric and meshless/algebraic) multigrid/multiresolution method for PDEs with rough ($L^\infty$) coefficients with rigorous a-priori accuracy and performance estimates. The method is discovered…
Let $A,$ $T$ and $B$ be bounded linear operators on a Banach space. This paper is concerned mainly with finding some necessary and sufficient conditions for convergence in operator norm of the sequences $\left\{ A^{n}TB^{n}\right\} $ and…
Given a multigrid procedure for linear systems with coefficient matrices $A_n$, we discuss the optimality of a related multigrid procedure with the same smoother and the same projector, when applied to properly related algebraic problems…
This paper studies accelerated gradient methods for nonconvex optimization with Lipschitz continuous gradient and Hessian. We propose two simple accelerated gradient methods, restarted accelerated gradient descent (AGD) and restarted heavy…
This paper reveals that a common and central role, played in many error bound (EB) conditions and a variety of gradient-type methods, is a residual measure operator. On one hand, by linking this operator with other optimality measures, we…
The accurate assembly of the system matrix is an important step in any code that solves partial differential equations on a mesh. We either explicitly set up a matrix, or we work in a matrix-free environment where we have to be able to…
Prior knowledge on properties of a target model often come as discrete or combinatorial descriptions. This work provides a unified computational framework for defining norms that promote such structures. More specifically, we develop…
Mechanical systems are often characterized only by their response to certain loads known from experiments or simulations. The obtained data can be used for various purposes: system analysis, design of mathematical models, or construction of…
Composite minimization is a powerful framework in large-scale convex optimization, based on decoupling of the objective function into terms with structurally different properties and allowing for more flexible algorithmic design. We…
Adaptive gradient methods have become popular in optimizing deep neural networks; recent examples include AdaGrad and Adam. Although Adam usually converges faster, variations of Adam, for instance, the AdaBelief algorithm, have been…
We revise the operator-norm convergence of the Trotter product formula for a pair {A,B} of generators of semigroups on a Banach space. Operator-norm convergence holds true if the dominating operator A generates a holomorphic contraction…
In this note, we show a sublinear nonergodic convergence rate for the algorithm developed in [Bai, et al. Generalized symmetric ADMM for separable convex optimization. Comput. Optim. Appl. 70, 129-170 (2018)], as well as its linear…
In this paper, we consider a classical form of optimal algebraic multigrid (AMG) interpolation that directly minimizes the two-grid convergence rate and compare it with the so-called ideal form that minimizes a certain weak approximation…
Adaptive gradient methods are workhorses in deep learning. However, the convergence guarantees of adaptive gradient methods for nonconvex optimization have not been thoroughly studied. In this paper, we provide a fine-grained convergence…
Parametric Gr\"obner bases have been studied for more than 15 years and are now a further developed subject. Here we propose a general study of parametric standard bases, that is with local orders. We mainly focus on the commutative case…
The convergence of multigrid methods degrades significantly if a small number of low quality cells are present in a finite element mesh, and this can be a barrier to the efficient and robust application of multigrid on complicated geometric…
Let $T$ be a weakly almost periodic (WAP) linear operator on a Banach space $X$. A sequence of scalars $(a_n)_{n\ge 1}$ {\it modulates} $T$ on $Y \subset X$ if $\frac1n\sum_{k=1}^n a_kT^k x$ converges in norm for every $x \in Y$. We obtain…
The paper studies a bounded symmetric operator ${\mathbf{A}}_\varepsilon$ in $L_2(\mathbf{R}^d)$ with $$ ({\mathbf{A}}_\varepsilon u) (x) = \varepsilon^{-d-2} \int_{\mathbf{R}^d} a((x-y)/\varepsilon) \mu(x/\varepsilon, y/\varepsilon) \left(…
We consider continuous Dirac operators defined on $\mathbf{R}^d$, $d\in\{1,2,3\}$, together with various discrete versions of them. Both forward-backward and symmetric finite differences are used as approximations to partial derivatives. We…
Adam is one of the most influential adaptive stochastic algorithms for training deep neural networks, which has been pointed out to be divergent even in the simple convex setting via a few simple counterexamples. Many attempts, such as…