Related papers: Derivative-free discrete gradient methods
We present a stochastic descent algorithm for unconstrained optimization that is particularly efficient when the objective function is slow to evaluate and gradients are not easily obtained, as in some PDE-constrained optimization and…
In this paper we construct a third order method for solving additively split autonomous stiff systems of ordinary differential equations. The constructed additive method is L-stable with respect to the implicit part and allows to use an…
A notion of implicit difference equation on a Lie groupoid is introduced and an algorithm for extracting the integrable part (backward or/and forward) is formulated. As an application, we prove that discrete Lagrangian dynamics on a Lie…
This article proposes new perspectives for developing derivative based numerical algorithms, supported by the introduction of a generalized derivative operators. It demonstrates that these operators have the potential to enhance and extend…
We provide a comprehensive survey of splitting and composition methods for the numerical integration of ordinary differential equations (ODEs). Splitting methods constitute an appropriate choice when the vector field associated with the ODE…
This paper deals with stochastic optimization problems involving Markovian noise with a zero-order oracle. We present and analyze a novel derivative-free method for solving such problems in strongly convex smooth and non-smooth settings…
In this paper, we will provide an introduction to the derivative-free optimization algorithms which can be potentially applied to train deep learning models. Existing deep learning model training is mostly based on the back propagation…
Neural Ordinary Differential Equations (Neural ODEs) are the continuous analog of Residual Neural Networks (ResNets). We investigate whether the discrete dynamics defined by a ResNet are close to the continuous one of a Neural ODE. We first…
Gradients have been exploited in proposal distributions to accelerate the convergence of Markov chain Monte Carlo algorithms on discrete distributions. However, these methods require a natural differentiable extension of the target discrete…
The numerical solution methods for partial differential equation (PDE) solution allow obtaining a discrete field that converges towards the solution if the method is applied to the correct problem. Nevertheless, the numerical methods…
We focus on analyzing the classical stochastic projected gradient methods under a general dependent data sampling scheme for constrained smooth nonconvex optimization. We show the worst-case rate of convergence $\tilde{O}(t^{-1/4})$ and…
Conventional finite-difference schemes for solving partial differential equations are based on approximating derivatives by finite-differences. In this work, an alternative theory is proposed which view finite-difference schemes as…
Inverse problems are ubiquitous because they formalize the integration of data with mathematical models. In many scientific applications the forward model is expensive to evaluate, and adjoint computations are difficult to employ; in this…
A discrete rotation algorithm can be apprehended as a parametric application $f\_\alpha$ from $\ZZ[i]$ to $\ZZ[i]$, whose resulting permutation ``looks like'' the map induced by an Euclidean rotation. For this kind of algorithm, to be…
All Darboux integrable difference equations on the quad-graph are described in the case of the equations that possess autonomous first-order integrals in one of the characteristics. A generalization of the discrete Liouville equation is…
We propose a new framework to design and analyze accelerated methods that solve general monotone equation (ME) problems $F(x)=0$. Traditional approaches include generalized steepest descent methods and inexact Newton-type methods. If $F$ is…
We explore a family of numerical methods, based on the Steffensen divided difference iterative algorithm, that do not evaluate the derivative of the objective functions. The family of methods achieves second-order convergence with two…
We are interested in derivative-free optimization of high-dimensional functions. The sample complexity of existing methods is high and depends on problem dimensionality, unlike the dimensionality-independent rates of first-order methods.…
Recently, Mboup, Join and Fliess [27], [28] introduced non-asymptotic integer order differentiators by using an algebraic parametric estimation method [7], [8]. In this paper, in order to obtain non-asymptotic fractional order…
We develop a monotone, two-scale discretization for a class of integrodifferential operators of order $2s$, $s \in (0,1)$. We apply it to develop numerical schemes, and derive pointwise convergence rates, for linear and obstacle problems…