Related papers: Using generalized simplex methods to approximate d…
An explicit formula to approximate the diagonal entries of the Hessian is introduced. When the derivative-free technique called \emph{generalized centered simplex gradient} is used to approximate the gradient, then the formula can be…
This work presents a novel matrix-based method for constructing an approximation Hessian using only function evaluations. The method requires less computational power than interpolation-based methods and is easy to implement in matrix-based…
This work introduces the nested-set Hessian approximation, a second-order approximation method that can be used in any derivative-free optimization routine that requires such information. It is built on the foundation of the generalized…
In this paper, we analyze several methods for approximating gradients of noisy functions using only function values. These methods include finite differences, linear interpolation, Gaussian smoothing and smoothing on a sphere. The methods…
This work investigates finite differences and the use of interpolation models to obtain approximations to the first and second derivatives of a function. Here, it is shown that if a particular set of points is used in the interpolation…
Gradient approximations are a class of numerical approximation techniques that are of central importance in numerical optimization. In derivative-free optimization, most of the gradient approximations, including the simplex gradient,…
We apply the Simple Equations Method (SEsM) for obtaining exact solutions of nonlinear differential equations. We discuss several examples with goal to illustrate the results from the use of derivatives of composite functions in the…
A generalized Gaussian process model (GGPM) is a unifying framework that encompasses many existing Gaussian process (GP) models, such as GP regression, classification, and counting. In the GGPM framework, the observation likelihood of the…
Many problems of theoretical and practical interest involve finding a convex or concave function. For instance, optimization problems such as finding the projection on the convex functions in $H^k(\Omega)$, or some problems in economics. In…
We propose an equivalent formula for the higher-order derivatives used in the study of Generalized Almost Perfect Nonlinear functions over an arbitrary finite field of characteristic $p$. The result is obtained by counting the number of…
A new method for approximating fractional derivatives of the Gaussian function and Dawson's integral are presented. Unlike previous approaches, which are dominantly based on some discretization of Riemann-Liouville integral using polynomial…
Several machine learning applications involve the optimization of higher-order derivatives (e.g., gradients of gradients) during training, which can be expensive in respect to memory and computation even with automatic differentiation. As a…
The optimal value function is one of the basic objects in the field of mathematical optimization, as it allows the evaluation of the variations in the cost/revenue generated while minimizing/maximizing a given function under some…
A new method to represent and approximate rotation matrices is introduced. The method represents approximations of a rotation matrix $Q$ with linearithmic complexity, i.e. with $\frac{1}{2}n\lg(n)$ rotations over pairs of coordinates,…
The Laplace approximation calls for the computation of second derivatives at the likelihood maximum. When the maximum is found by the EM-algorithm, there is a convenient way to compute these derivatives. The likelihood gradient can be…
In this paper, we consider an approximation method, and a novel general analysis, for second-order elliptic differential equations with heterogeneous multiscale coefficients. We obtain convergence of the Generalized Multi-scale Finite…
This article proposes new perspectives for developing derivative based numerical algorithms, supported by the introduction of a generalized derivative operators. It demonstrates that these operators have the potential to enhance and extend…
In this work we develop Curvature Propagation (CP), a general technique for efficiently computing unbiased approximations of the Hessian of any function that is computed using a computational graph. At the cost of roughly two gradient…
In Mergelyan type approximation we uniformly approximate functions on compact sets K by polynomials or rational functions or holomorphic functions on varying open sets containing K. In the present paper we consider analogous approximation,…
This paper presents an algorithm to simulate Gaussian random vectors whose precision matrix can be expressed as a polynomial of a sparse matrix. This situation arises in particular when simulating Gaussian Markov random fields obtained by…