Related papers: Improving Efficiency of Parallel Across the Method…

Finite differences and Runge-Kutta time stepping schemes used in Computational AeroAcoustics simulations are often optimized for low dispersion and dissipation (e.g. DRP or LDDRK schemes) when applied to linear problems in order to…

We introduce a new, high-throughput, synchronous, distributed, data-parallel, stochastic-gradient-descent learning algorithm. This algorithm uses amortized inference in a compute-cluster-specific, deep, generative, dynamical model to…

In this paper we develop random block coordinate gradient descent methods for minimizing large scale linearly constrained separable convex problems over networks. Since we have coupled constraints in the problem, we devise an algorithm that…

This paper presents a sequential convex programming (SCP) framework for ensuring the continuous-time satisfaction of compound state-triggered constraints, a subset of logical specifications, in the powered descent guidance (PDG) problem.…

Solving evolutionary equations in a parallel-in-time manner is an attractive topic and many algorithms are proposed in recent two decades. The algorithm based on the block $\alpha$-circulant preconditioning technique has shown promising…

System performance for networks composed of interconnected subsystems can be increased if the traditionally separated subsystems are jointly optimized. Recently, parallel and distributed optimization methods have emerged as a powerful tool…

Asynchronous stochastic gradient descent (ASGD) is a popular parallel optimization algorithm in machine learning. Most theoretical analysis on ASGD take a discrete view and prove upper bounds for their convergence rates. However, the…

Mathematical programs with complementarity constraints (MPCCs) are a challenging class of nonlinear optimization problems, because their nonlinear programming reformulations violate standard constraint qualifications at every feasible…

We revisit the numerical stability of four well-established explicit stochastic integration schemes through a new generic benchmark stochastic differential equation designed to assess asymptotic statistical accuracy and stability…

We propose a micro-macro parallel-in-time Parareal method for scalar McKean-Vlasov stochastic differential equations (SDEs). In the algorithm, the fine Parareal propagator is a Monte Carlo simulation of an ensemble of particles, while an…

In recent years, many estimation problems in robotics have been shown to be solvable to global optimality using their semidefinite relaxations. However, the runtime complexity of off-the-shelf semidefinite programming (SDP) solvers is up to…

We study the construction and convergence of semi-explicit and iterative decoupling schemes for an elliptic-parabolic problem using higher-order Runge-Kutta methods. For the semi-explicit schemes, which are constructed using a nearby delay…

Mixed precision Runge--Kutta methods have been recently developed and used for the time-evolution of partial differential equations. Two-derivative Runge--Kutta schemes may offer enhanced stability and accuracy properties compared to…

We develop and analyze new scheduling algorithms for solving sparse triangular linear systems (SpTRSV) in parallel. Our approach produces highly efficient synchronous schedules for the forward- and backward-substitution algorithm. Compared…

This paper presents a highly-parallelizable parallel-in-time algorithm for efficient solution of nonlinear time-periodic problems. It is based on the time-periodic extension of the Parareal method, known to accelerate sequential…

The spectral deferred correction method is a variant of the deferred correction method for solving ordinary differential equations. A benefit of this method is that is uses low order schemes iteratively to produce a high order…

Stochastic gradient methods (SGMs) are the predominant approaches to train deep learning models. The adaptive versions (e.g., Adam and AMSGrad) have been extensively used in practice, partly because they achieve faster convergence than the…

Supercomputers are equipped with an increasingly large number of cores to use computational power as a way of solving problems that are otherwise intractable. Unfortunately, getting serial algorithms to run in parallel to take advantage of…

Semidefinite programs (SDPs) are a fundamental class of optimization problems with important recent applications in approximation algorithms, quantum complexity, robust learning, algorithmic rounding, and adversarial deep learning. This…

LLM serving platforms are increasingly deployed as multi-model cloud systems, where user demand is often long-tailed: a few popular large models receive most requests, while many smaller tail models remain underutilized. We propose…