Related papers: Numerical Stability of s-step Enlarged Krylov Subs…
Krylov subspace methods are a powerful tool for efficiently solving high-dimensional linear algebra problems. In this work, we study the approximation quality that a Krylov subspace provides for estimating the numerical range of a matrix.…
Asynchronous computation and gradient compression have emerged as two key techniques for achieving scalability in distributed optimization for large-scale machine learning. This paper presents a unified analysis framework for distributed…
This paper investigates the problems large-scale distributed composite convex optimization, with motivations from a broad range of applications, including multi-agent systems, federated learning, smart grids, wireless sensor networks,…
In this paper we present a novel extended Krylov subspace reduced-order modeling technique to efficiently simulate time- and frequency-domain wavefields in open complex structures. To simulate the extension to infinity, we use an optimal…
Bilevel optimization, with broad applications in machine learning, has an intricate hierarchical structure. Gradient-based methods have emerged as a common approach to large-scale bilevel problems. However, the computation of the…
The purpose of this paper is to introduce a new Kirk type iterative algorithm called Kirk multistep iteration and to study its convergence. We also prove some theorems related with the stability results for the Kirk-multistep and Kirk-SP…
In fluid dynamics, predicting and characterizing bifurcations, from the onset of unsteadiness to the transition to turbulence, is of critical importance for both academic and industrial applications. Different tools from dynamical systems…
To reduce the long training time of large deep neural network (DNN) models, distributed synchronous stochastic gradient descent (S-SGD) is commonly used on a cluster of workers. However, the speedup brought by multiple workers is limited by…
We introduce novel convergence results for asynchronous iterations that appear in the analysis of parallel and distributed optimization algorithms. The results are simple to apply and give explicit estimates for how the degree of asynchrony…
We prove new convergence rates for a generalized version of stochastic Nesterov acceleration under interpolation conditions. Unlike previous analyses, our approach accelerates any stochastic gradient method which makes sufficient progress…
Many scientific applications require the evaluation of the action of the matrix function over a vector and the most common methods for this task are those based on the Krylov subspace. Since the orthogonalization cost and memory requirement…
This paper focuses on the construction and analysis of explicit numerical methods of high dimensional stochastic nonlinear Schrodinger equations (SNLSEs). We first prove that the classical explicit numerical methods are unstable and suffer…
In this work, we propose and develop efficient and accurate numerical methods for solving the Kirchhoff-Love plate model in domains with complex geometries. The algorithms proposed here employ curvilinear finite-difference methods for…
A novel numerical approach to solving the shallow-water equations on the sphere using high-order numerical discretizations in both space and time is proposed. A space-time tensor formalism is used to express the equations of motion…
The computation of the entire Lyapunov spectrum for extended dynamical systems is a very time consuming task. If the system is in a chaotic spatio-temporal regime it is possible to approximately reconstruct the Lyapunov spectrum from the…
Distributed stochastic gradient descent (SGD) is essential for scaling the machine learning algorithms to a large number of computing nodes. However, the infrastructures variability such as high communication delay or random node slowdown…
Although it is relatively easy to apply, the gradient method often displays a disappointingly slow rate of convergence. Its convergence is specially based on the structure of the matrix of the algebraic linear system, and on the choice of…
Smoothing accelerated gradient methods achieve faster convergence rates than that of the subgradient method for some nonsmooth convex optimization problems. However, Nesterov's extrapolation may require gradients at infeasible points, and…
Vector extrapolation methods are widely used in large-scale simulation studies, and numerous extrapolation-based acceleration techniques have been developed to enhance the convergence of linear and nonlinear fixed-point iterative methods.…
Inclusion of a term $-\gamma\nabla\nabla\cdot u$, forcing $\nabla\cdot u$ to be pointwise small, is an effective tool for improving mass conservation in discretizations of incompressible flows. However, the added grad-div term couples all…