Related papers: Discrete gradient algorithms of high order for one…
We study a fixed step-size noisy distributed gradient descent algorithm for solving optimization problems in which the objective is a finite sum of smooth but possibly non-convex functions. Random perturbations are introduced to the…
We develop a general framework for numerically solving differential equations while preserving invariants. As in standard projection methods, we project an arbitrary base integrator onto an invariant-preserving manifold, however, our method…
Efforts to achieve better accuracy in numerical relativity have so far focused either on implementing second order accurate adaptive mesh refinement or on defining higher order accurate differences and update schemes. Here, we argue for the…
Exponential integrators based on contour integral representations lead to powerful numerical solvers for a variety of ODEs, PDEs, and other time-evolution equations. They are embarrassingly parallelizable and lead to global-in-time…
We show that when time-reversible symplectic algorithms are used to solve periodic motions, the energy error after one period is generally two orders higher than that of the algorithm. By use of correctable algorithms, we show that the…
In this paper, an adjustment to the original differentially private stochastic gradient descent (DPSGD) algorithm for deep learning models is proposed. As a matter of motivation, to date, almost no state-of-the-art machine learning…
Scientific studies often require the precise calculation of derivatives. In many cases an analytical calculation is not feasible and one resorts to evaluating derivatives numerically. These are error-prone, especially for higher-order…
This paper investigates the privacy-preserving distributed optimization problem, aiming to protect agents' private information from potential attackers during the optimization process. Gradient tracking, an advanced technique for improving…
This paper presents a high-order differentiator for delayed measurement signal. The proposed differentiator not only can correct the delay in signal, but aslo can estimate the undelayed derivatives. The differentiator consists of two-step…
In this paper, we introduce a Lagrange multiplier approach to construct linearly implicit energy-preserving schemes of arbitrary order for general Hamiltonian PDEs. Unlike the widely used auxiliary variable methods, this novel approach does…
We present sixth- and eighth-order Hermite integrators for astrophysical $N$-body simulations, which use the derivatives of accelerations up to second order ({\it snap}) and third order ({\it crackle}). These schemes do not require previous…
Gradient-descent based iterative algorithms pervade a variety of problems in estimation, prediction, learning, control, and optimization. Recently iterative algorithms based on higher-order information have been explored in an attempt to…
The inaccuracy of the classical magnetic field integral equation (MFIE) is a long-studied problem. We investigate one of the potential approaches to solve the accuracy problem: higher-order discretization schemes. While these are able to…
We present a family of multistep integrators based on the Adams-Bashforth methods. These schemes can be constructed for arbitrary convergence order with arbitrary step size variation. The step size can differ between different subdomains of…
This work establishes new convergence guarantees for gradient descent in smooth convex optimization via a computer-assisted analysis technique. Our theory allows nonconstant stepsize policies with frequent long steps potentially violating…
Most zeroth-order optimization algorithms mimic a first-order algorithm but replace the gradient of the objective function with some gradient estimator that can be computed from a small number of function evaluations. This estimator is…
Accelerated gradient descent iterations are widely used in optimization. It is known that, in the continuous-time limit, these iterations converge to a second-order differential equation which we refer to as the accelerated gradient flow.…
This work deals with the numerical approximation of plasmas which are confined by the effect of a fast oscillating magnetic field (see \cite{Bostan2012}) in the Vlasov model. The presence of this magnetic field induces oscillations (in…
We consider high-order splitting schemes for large-scale differential Riccati equations. Such equations arise in many different areas and are especially important within the field of optimal control. In the large-scale case, it is critical…
We construct new, efficient, and accurate high-order finite differencing operators which satisfy summation by parts. Since these operators are not uniquely defined, we consider several optimization criteria: minimizing the bandwidth, the…