Related papers: Higher-order Derivatives of Weighted Finite-state …
For a large class of variational quantum circuits, we show how arbitrary-order derivatives can be analytically evaluated in terms of simple parameter-shift rules, i.e., by running the same circuit with different shifts of the parameters. As…
Scientific studies often require the precise calculation of derivatives. In many cases an analytical calculation is not feasible and one resorts to evaluating derivatives numerically. These are error-prone, especially for higher-order…
We propose a simple recursive algorithm that allows the computation of the first- and second-order derivatives with respect to the inputs of an arbitrary deep feed forward neural network (DFNN). The algorithm naturally incorporates the…
In many scenarios, a state-space model depends on a parameter which needs to be inferred from data. Using stochastic gradient search and the optimal filter (first-order) derivative, the parameter can be estimated online. To analyze the…
This paper introduces a new computational framework to derive electromagnetic field derivatives with respect to multiple design parameters up to any order with the Finite-Difference Time-Domain (FDTD) technique. Specifically, only one FDTD…
Motivated by the problem of solving the Einstein equations, we discuss high order finite difference discretizations of first order in time, second order in space hyperbolic systems.Particular attention is paid to the case when first order…
First-order stochastic methods are the state-of-the-art in large-scale machine learning optimization owing to efficient per-iteration complexity. Second-order methods, while able to provide faster convergence, have been much less explored…
Time delays are a common perturbation in systems with many states, such as networked, distributed, or decentralized systems. Current methods analyzing the stability of large systems with time delay typically produce very conservative…
In this paper, we propose a new set of midpoint-based high-order discretization schemes for computing straight and mixed nonlinear second derivative terms that appear in the compressible Navier-Stokes equations. Firstly, we detail a set of…
We present an efficient finite difference method for the approximation of second derivatives, with respect to system parameters, of expectations for a class of discrete stochastic chemical reaction networks. The method uses a coupling of…
The first order derivative of a data density can be estimated efficiently by denoising score matching, and has become an important component in many applications, such as image generation and audio synthesis. Higher order derivatives…
Because of the nonlocal properties of fractional operators, higher order schemes play more important role in discretizing fractional derivatives than classical ones. The striking feature is that higher order schemes of fractional…
We propose a novel second-order optimization framework for training the emerging deep continuous-time models, specifically the Neural Ordinary Differential Equations (Neural ODEs). Since their training already involves expensive gradient…
Linear differential equations are ubiquitous in science and engineering. Quantum computers can simulate quantum systems, which are described by a restricted type of linear differential equations. Here we extend quantum simulation algorithms…
Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent. Second-order optimization methods, that involve second derivatives and/or second…
There exist many applications where it is necessary to approximate numerically derivatives of a function which is given by a computer procedure. In particular, all the fields of optimization have a special interest in such a kind of…
Factorization machines (FMs) are a supervised learning approach that can use second-order feature combinations even when the data is very high-dimensional. Unfortunately, despite increasing interest in FMs, there exists to date no efficient…
Stochastic processes find applications in modelling systems in a variety of disciplines. A large number of stochastic models considered are Markovian in nature. It is often observed that higher order Markov processes can model the data…
Finite-sum optimization problems are ubiquitous in machine learning, and are commonly solved using first-order methods which rely on gradient computations. Recently, there has been growing interest in \emph{second-order} methods, which rely…
We aim at computing the derivative of the solution to a parametric optimization problem with respect to the involved parameters. For a class broader than that of strongly convex functions, this can be achieved by automatic differentiation…