Related papers: SRKCD: a stabilized Runge-Kutta method for stochas…

In this paper we consider large-scale smooth optimization problems with multiple linear coupled constraints. Due to the non-separability of the constraints, arbitrary random sketching would not be guaranteed to work. Thus, we first…

The problem of solving stochastic differential-algebraic equations (SDAEs) of index one with a scalar driving Brownian motion is considered. Recently, the authors proposed a class of stiffly accurate stochastic Runge-Kutta (SRK) methods…

When applied to stiff, linear differential equations with time-dependent forcing, Runge-Kutta methods can exhibit convergence rates lower than predicted by the classical order condition theory. Commonly, this order reduction phenomenon is…

The work deals with two major topics concerning the numerical analysis of Runge-Kutta-like (RK-like) methods, namely their stability and order of convergence. RK-like methods differ from additive RK methods in that their coefficients are…

We explore a novel way to numerically resolve the scaling behavior of finite-time singularities in solutions of nonlinear parabolic PDEs. The Runge--Kutta--Legendre (RKL) and Runge--Kutta--Gegenbauer (RKG) super-time-stepping methods were…

Stochastic optimization algorithms with variance reduction have proven successful for minimizing large finite sums of functions. Unfortunately, these techniques are unable to deal with stochastic perturbations of input data, induced for…

Stochastic gradient descent (\textsc{Sgd}) methods are the most powerful optimization tools in training machine learning and deep learning models. Moreover, acceleration (a.k.a. momentum) methods and diagonal scaling (a.k.a. adaptive…

Classical convergence theory of Runge-Kutta methods assumes that the time step is small relative to the Lipschitz constant of the ordinary differential equation (ODE). For stiff problems, that assumption is often violated, and a problematic…

Sampling from a high-dimensional probability distribution is a fundamental algorithmic task arising in wide-ranging applications across multiple disciplines, including scientific computing, computational statistics and machine learning.…

Stochastic gradient methods have been a popular and powerful choice of optimization methods, aimed at minimizing functions. Their advantage lies in the fact that that one approximates the gradient as opposed to using the full Jacobian…

Finite differences and Runge-Kutta time stepping schemes used in Computational AeroAcoustics simulations are often optimized for low dispersion and dissipation (e.g. DRP or LDDRK schemes) when applied to linear problems in order to…

Stochastic gradient algorithms are often unstable when applied to functions that do not have Lipschitz-continuous and/or bounded gradients. Gradient clipping is a simple and effective technique to stabilize the training process for problems…

We study the connections between ordinary differential equations and optimization algorithms in a non-Euclidean setting. We propose a novel accelerated algorithm for minimising convex functions over a convex constrained set. This algorithm…

We investigate the Randomized Stochastic Accelerated Gradient (RSAG) method, utilizing either constant or adaptive step sizes, for stochastic optimization problems with generalized smooth objective functions. Under relaxed affine variance…

Extended Stability Runge-Kutta (ESRK) methods are crucial for solving large-scale computational problems in science and engineering, including weather forecasting, aerodynamic analysis, and complex biological modelling. However, balancing…

Many HPC applications that solve differential equations rely on the Runge-Kutta family of methods for time integration. Among these methods, the fourth-order accurate RK4 scheme is especially popular. This time integration scheme requires…

We develop universal gradient methods for Stochastic Convex Optimization (SCO). Our algorithms automatically adapt not only to the oracle's noise but also to the H\"older smoothness of the objective function without a priori knowledge of…

Coordinate-type subgradient methods for addressing nonsmooth optimization problems are relatively underexplored due to the set-valued nature of the subdifferential. In this work, our study focuses on nonsmooth composite optimization…

In this work we present a new class of Runge-Kutta (RK) methods for solving systems of hyperbolic equations with a particular structure, generalization of a wave-equation. The new methods are {\it partially implicit} in the sense that a…

In this paper, Runge-Kutta-Gegenbauer (RKG) stability polynomials of arbitrarily high order of accuracy are introduced in closed form. The stability domain of RKG polynomials extends in the the real direction with the square of polynomial…