Related papers: Asynchronous Time-Parallel Method based on Laplace…
The time integration of semilinear parabolic problems by exponential methods of different kinds is considered. A new algorithm for the implementation of these methods is proposed. The algorithm evaluates the operators required by the…
This paper proposes a parallel in time (called also time parareal) method to solve Volterra integral equations of the second kind. The parallel in time approach follows the same spirit as the domain decomposition that consists of breaking…
We present a novel approach to the parallelization of the parabolic fast multipole method for a space-time boundary element method for the heat equation. We exploit the special temporal structure of the involved operators to provide an…
This paper introduces the multiplicative variant of the recently proposed asynchronous additive coarse-space correction method. Definition of an asynchronous extension of multiplicative correction is not straightforward, however, our…
The current algorithms are based on linear model, for example, Precision Time Protocol (PTP) which requires frequent synchronization in order to handle the effects of clock frequency drift. This paper introduces a nonlinear approach to…
The purpose of this paper is to propose a semi-analytical technique convenient for numerical approximation of solutions of the initial value problem for $p$-dimensional delayed and neutral differential systems with constant, proportional…
We present a parallelized primal-dual algorithm for solving constrained convex optimization problems. The algorithm is "block-based," in that vectors of primal and dual variables are partitioned into blocks, each of which is updated only by…
Existing asynchronous distributed optimization algorithms often use diminishing step-sizes that cause slow practical convergence, or use fixed step-sizes that depend on and decrease with an upper bound of the delays. Not only are such delay…
A parallel algorithm for computing the finite difference solution to the elliptic equations with non-separable variables is presented. The resultant matrix is symmetric positive definite, thus the preconditioning conjugate gradient or the…
In this work we show that randomized (block) coordinate descent methods can be accelerated by parallelization when applied to the problem of minimizing the sum of a partially separable smooth convex function and a simple separable convex…
Sequential numerical methods for integrating initial value problems (IVPs) can be prohibitively expensive when high numerical accuracy is required over the entire interval of integration. One remedy is to integrate in a parallel fashion,…
The time parallel solution of optimality systems arising in PDE constraint optimization could be achieved by simply applying any time parallel algorithm, such as Parareal, to solve the forward and backward evolution problems arising in the…
In this paper, we introduce a novel semi-analytical method for solving a broad class of initial value problems involving differential, integro-differential, and delay equations, including those with fractional and variable-order…
Pipeline parallelism enables training models that exceed single-device memory, but practical throughput remains limited by pipeline bubbles. Although parameter freezing can improve training throughput by adaptively skipping backward…
Efficient parallelism is necessary for achieving low-latency, high-throughput inference with large language models (LLMs). Tensor parallelism (TP) is the state-of-the-art method for reducing LLM response latency, however GPU communications…
The so-called method of phase synchronization has been advocated in a number of papers as a way of decoupling a system of linear second-order differential equations by a linear transformation of coordinates and velocities. This is a rather…
We present a time-parallelization method that enables to accelerate the computation of quantum optimal control algorithms. We show that this approach is approximately fully efficient when based on a gradient method as optimization solver:…
In this article, a new modified Laplace-Fourier method is developed in order to obtain the solutions of linear neutral delay differential equations. The proposed method provides a more accurate solution than the one provided by the pure…
In this paper, we consider the convergence of a very general asynchronous-parallel algorithm called ARock, that takes many well-known asynchronous algorithms as special cases (gradient descent, proximal gradient, Douglas Rachford, ADMM,…
Stochastic gradient methods (SGMs) are the predominant approaches to train deep learning models. The adaptive versions (e.g., Adam and AMSGrad) have been extensively used in practice, partly because they achieve faster convergence than the…