Related papers: Discrete gradient algorithms of high order for one…
Context: Calculating stellar pulsations requires a sufficient accuracy to match the quality of the observations. Many current pulsation codes apply a second order finite-difference scheme, combined with Richardson extrapolation to reach…
Specialized function gradient computing hardware could greatly improve the performance of state-of-the-art optimization algorithms, e.g., based on gradient descent or conjugate gradient methods that are at the core of control, machine…
In this paper, we establish new convergence results for the quantized distributed gradient descent and suggest a novel strategy of choosing the stepsizes for the high-performance of the algorithm. Under the strongly convexity assumption on…
In this paper, we introduce a novel algorithm for calculating arbitrary order cumulants of multidimensional data. Since the $d^\text{th}$ order cumulant can be presented in the form of an $d$-dimensional tensor, the algorithm is presented…
The discrete-dipole approximation (DDA) is a flexible technique for computing scattering and absorption by targets of arbitrary geometry. In this paper we perform systematic study of various non-stationary iterative (conjugate gradient)…
A first-order linear fully discrete scheme is studied for the incompressible time-dependent Navier-Stokes equations in three-dimensional domains. This scheme, based on an incremental pressure projection method, decouples each component of…
This paper explores the relationship between numerical integrators and optimal control algorithms. Specifically, the performance of the differential dynamical programming (DDP) algorithm is examined when a variational integrator and a newly…
In this work, in order to obtain higher-order schemes for solving forward backward stochastic differential equations, we adopt the high-order multi-step method in [W. Zhao, Y. Fu and T. Zhou, SIAM J. Sci. Comput., 36(4) (2014),…
Discrete maximum principles in the approximation of partial differential equations are crucial for the preservation of qualitative properties of physical models. In this work we enforce the discrete maximum principle by performing a simple…
Higher-order exceptional points in non-Hermitian systems have recently been used as a tool to engineer high-sensitivity devices, attracting tremendous attention from multidisciplinary fields. Here, we present a simple yet effective scheme…
The aim of this paper is the derivation of structure preserving schemes for the solution of the EPDiff equation, with particular emphasis on the two dimensional case. We develop three different schemes based on the Discrete Variational…
We consider the numerical integration of non-autonomous separable parabolic equations using high order splitting methods with complex coefficients (methods with real coefficients of order greater than two necessarily have negative…
Privacy protection has become an increasingly pressing requirement in distributed optimization. However, equipping distributed optimization with differential privacy, the state-of-the-art privacy protection mechanism, will unavoidably…
The depth of networks plays a crucial role in the effectiveness of deep learning. However, the memory requirement for backpropagation scales linearly with the number of layers, which leads to memory bottlenecks during training. Moreover,…
Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent. Second-order optimization methods, that involve second derivatives and/or second…
First-order accurate degenerate variational integration (DVI) was introduced in C. L. Ellison et. al, Phys. Plasmas 25, 052502 (2018) for systems with a degenerate Lagrangian, i.e. one in which the velocity-space Hessian is singular. In…
The Sinkhorn operator has recently experienced a surge of popularity in computer vision and related fields. One major reason is its ease of integration into deep learning frameworks. To allow for an efficient training of respective neural…
In this paper, we propose a class of stochastic exponential discrete gradient schemes for SDEs with linear and gradient components in the coefficients. The root mean-square errors of the schemes are analyzed, and the structure-preserving…
We show that accelerated optimization methods can be seen as particular instances of multi-step integration schemes from numerical analysis, applied to the gradient flow equation. In comparison with recent advances in this vein, the…
Interpreting gradient methods as fixed-point iterations, we provide a detailed analysis of those methods for minimizing convex objective functions. Due to their conceptual and algorithmic simplicity, gradient methods are widely used in…