Related papers: Efficient Higher Order Derivatives of Objective Fu…
We derive algorithms for higher order derivative computation of the rectangular $QR$ and eigenvalue decomposition of symmetric matrices with distinct eigenvalues in the forward and reverse mode of algorithmic differentiation (AD) using…
Optimizing neural networks with loss that contain high-dimensional and high-order differential operators is expensive to evaluate with back-propagation due to $\mathcal{O}(d^{k})$ scaling of the derivative tensor size and the…
Computing the matrix square root or its inverse in a differentiable manner is important in a variety of computer vision tasks. Previous methods either adopt the Singular Value Decomposition (SVD) to explicitly factorize the matrix or use…
We address the task of higher-order derivative evaluation of computer programs that contain QR decompositions and real symmetric eigenvalue decompositions. The approach is a combination of univariate Taylor polynomial arithmetic and matrix…
Scientific studies often require the precise calculation of derivatives. In many cases an analytical calculation is not feasible and one resorts to evaluating derivatives numerically. These are error-prone, especially for higher-order…
Computing the matrix square root and its inverse in a differentiable manner is important in a variety of computer vision tasks. Previous methods either adopt the Singular Value Decomposition (SVD) to explicitly factorize the matrix or use…
Higher order derivatives of functions are structured high dimensional objects which lend themselves to many alternative representations, with the most popular being multi-index, matrix and tensor representations. The choice between them…
Taylor's formula holds significant importance in function representation, such as solving differential difference equations, ordinary differential equations, partial differential equations, and further promotes applications in visual…
It is commonly assumed that calculating third order information is too expensive for most applications. But we show that the directional derivative of the Hessian ($D^3f(x)\cdot d$) can be calculated at a cost proportional to that of a…
We consider Hadamard fractional derivatives and integrals of variable fractional order. A new type of fractional operator, which we call the Hadamard-Marchaud fractional derivative, is also considered. The objective is to represent these…
Perturbation or error bounds of functions have been of great interest for a long time. If the functions are differentiable, then the mean value theorem and Taylor's theorem come handy for this purpose. While the former is useful in…
In this work we use the tensorial language developed in [8] and [9] to differentiate functions of eigenvalues of symmetric matrices. We describe the formulae for the k-th derivative of such functions in two cases. The first case concerns…
We introduce and extend the outer product and contractive product of tensors and matrices, and present some identities in terms of these products. We offer tensor expressions of derivatives of tensors, focus on the tensor forms of…
In this work we propose a generalization of the Hadamard product between two matrices to a tensor-valued, multi-linear product between k matrices for any $k \ge 1$. A multi-linear dual operator to the generalized Hadamard product is…
A method to increase the precision of feedforward networks is proposed. It requires a prior knowledge of a target function derivatives of several orders and uses this information in gradient based training. Forward pass calculates not only…
This paper introduces a new computational framework to derive electromagnetic field derivatives with respect to multiple design parameters up to any order with the Finite-Difference Time-Domain (FDTD) technique. Specifically, only one FDTD…
When training large models, such as neural networks, the full derivatives of order 2 and beyond are usually inaccessible, due to their computational cost. Therefore, among the second-order optimization methods, it is common to bypass the…
This paper proposes a Direct Rational Radial Basis Functions Partition of Unity (D-RRBF-PU) approach to compute derivatives of functions with steep gradients or discontinuities. The novelty of the method concerns how derivatives are…
Suitable discretizations through tensor product formulas of popular multidimensional operators (diffusion or diffusion--advection, for instance) lead to matrices with $d$-dimensional Kronecker sum structure. For evolutionary Partial…
This paper introduces a new approach for the computation of electromagnetic field derivatives, up to any order, with respect to the material and geometric parameters of a given geometry, in a single Finite-Difference Time-Domain (FDTD)…