Related papers: Derivative-free discrete gradient methods
Finite-difference methods are a class of algorithms designed to solve black-box optimization problems by approximating a gradient of the target function on a set of directions. In black-box optimization, the non-smooth setting is…
In this paper, we propose new structured second-order methods and structured adaptive-gradient methods obtained by performing natural-gradient descent on structured parameter spaces. Natural-gradient descent is an attractive approach to…
In this paper, we analyze several methods for approximating gradients of noisy functions using only function values. These methods include finite differences, linear interpolation, Gaussian smoothing and smoothing on a sphere. The methods…
A novel discontinuous Galerkin (DG) method is developed to solve time-dependent bi-harmonic type equations involving fourth derivatives in one and multiple space dimensions. We present the spatial DG discretization based on a mixed…
Variational inference in Bayesian deep learning often involves computing the gradient of an expectation that lacks a closed-form solution. In these cases, pathwise and score-function gradient estimators are the most common approaches. The…
In the present paper, by approximating the derivatives in the Kou et al. \cite{Kou} fourth-order method by central difference quotient, we obtain new modification of this method free from derivatives. We prove the important fact that the…
We show a novel systematic way to construct conservative finite difference schemes for quasilinear first-order system of ordinary differential equations with conserved quantities. In particular, this includes both autonomous and…
We propose a new class of rigorous methods for derivative-free optimization with the aim of delivering efficient and robust numerical performance for functions of all types, from smooth to non-smooth, and under different noise regimes. To…
Recent studies have shown that fractional calculus is an effective alternative mathematical tool in various scientific fields. However, some investigations indicate that results established in differential and integral calculus do not…
A piecewise constant Mayer cost function is used to model optimal control problems in which the state space is partitioned into several regions, each having its own Mayer cost value. In such a context, the standard numerical methods used in…
This study investigates numerical methods to solve nonlinear transport problems characterized by various sorption isotherms with a focus on the Freundlich type of isotherms. We describe and compare second order accurate numerical schemes,…
A study is conducted to evaluate four derivative estimation methods when solving a large sparse nonlinear programming problem that arises from the approximation of an optimal control problem using a direct collocation method. In particular,…
In this paper we deal with a general second order continuous dynamical system associated to a convex minimization problem with a Fr\`echet differentiable objective function. We show that inertial algorithms, such as Nesterov's algorithm,…
Single-stage or single-step high-order temporal discretizations of partial differential equations (PDEs) have shown great promise in delivering high-order accuracy in time with efficient use of computational resources. There has been much…
In the framework of ODEs, we uncover a new link between the continuous Galerkin method (see Math. Comp. (1972), 26 (118 and 120), 415-426 and 881-891) and the discontinuous Galerkin method (see Mathematical Aspects of Finite elements in…
In this work, we present an efficient gradient projection method for solving a class of stochastic optimal control problem with expected integral state constraint. The first order optimality condition system consisting of forward-backward…
Automatic differentiation is everywhere, but there exists only minimal documentation of how it works in complex arithmetic beyond stating "derivatives in $\mathbb{C}^d$" $\cong$ "derivatives in $\mathbb{R}^{2d}$" and, at best, shallow…
We propose a general strategy to discretize the Dyson series without applying direct numerical quadrature to high-dimensional integrals, and extend this framework to open quantum systems. The resulting discretization can also be interpreted…
The existing discrete variational derivative method is only second-order accurate and fully implicit. In this paper, we propose a framework to construct an arbitrary high-order implicit (original) energy stable scheme and a second-order…
We address composite optimization problems, which consist in minimizing the sum of a smooth and a merely lower semicontinuous function, without any convexity assumptions. Numerical solutions of these problems can be obtained by proximal…