Related papers: Efficient Automatic Differentiation of Implicit Fu…

We introduce an algorithm that simplifies the construction of efficient estimators, making them accessible to a broader audience. 'Dimple' takes as input computer code representing a parameter of interest and outputs an efficient estimator.…

Machine learning and neural network models in particular have been improving the state of the art performance on many artificial intelligence related tasks. Neural network models are typically implemented using frameworks that perform…

In this paper we take a look at Automatic Differentiation through the eyes of Tensor and Operational Calculus. This work is best consumed as supplementary material for learning tensor and operational calculus by those already familiar with…

The computational burden of probabilistic inference remains a hurdle for applying probabilistic programming languages to practical problems of interest. In this work, we provide a semantic and algorithmic foundation for efficient exact…

We prove a closed formula for the derivative, of any order, of a implicit function, in terms of some binomial building blocks, and explain the combinatorics behind the coefficients appearing in the formula.

The technique of abstracting abstract machines (AAM) provides a systematic approach for deriving computable approximations of evaluators that are easily proved sound. This article contributes a complementary step-by-step process for…

Ou et al. (2022) introduce the problem of learning set functions from data generated by a so-called optimal subset oracle. Their approach approximates the underlying utility function with an energy-based model, whose parameters are…

Probabilistic modeling is iterative. A scientist posits a simple model, fits it to her data, refines it according to her analysis, and repeats. However, fitting complex models to large data is a bottleneck in this process. Deriving…

The Integral Image algorithm is often applied in tasks that require efficient integration over images, such as object detection. In this paper we discuss theoretical aspects of the algorithm's continuous version. We suggest to define the…

The adjoint method is an efficient way to numerically compute gradients in optimization problems with constraints, but is only formulated to differentiable cost and constraint functions on real variables. With the introduction of complex…

In this work, we discuss the Automatic Adjoint Differentiation (AAD) for functions of the form $G=\frac{1}{2}\sum_1^m (Ey_i-C_i)^2$, which often appear in the calibration of stochastic models. { We demonstrate that it allows a perfect…

In scientific computation, it is often necessary to calculate higher-order derivatives of a function. Currently, two primary methods for higher-order automatic differentiation exist: symbolic differentiation and algorithmic automatic…

Kaltofen has proposed a new approach in [Kaltofen 1992] for computing matrix determinants. The algorithm is based on a baby steps/giant steps construction of Krylov subspaces, and computes the determinant as the constant term of a…

Recently, we have proposed a new diffusive representation for fractional derivatives and, based on this representation, suggested an algorithm for their numerical computation. From the construction of the algorithm, it is immediately…

Differentiation along algorithms, i.e., piggyback propagation of derivatives, is now routinely used to differentiate iterative solvers in differentiable programming. Asymptotics is well understood for many smooth problems but the…

In this paper, we present a method for the accurate estimation of the derivative (aka.~sensitivity) of expectations of functions involving an indicator function by combining a stochastic algorithmic differentiation and a regression. The…

Many engineering problems involve learning hidden dynamics from indirect observations, where the physical processes are described by systems of partial differential equations (PDE). Gradient-based optimization methods are considered…

We aim at computing the derivative of the solution to a parametric optimization problem with respect to the involved parameters. For a class broader than that of strongly convex functions, this can be achieved by automatic differentiation…

Conventional finite-difference schemes for solving partial differential equations are based on approximating derivatives by finite-differences. In this work, an alternative theory is proposed which view finite-difference schemes as…

First-order optimization algorithms, often preferred for large problems, require the gradient of the differentiable terms in the objective function. These gradients often involve linear operators and their adjoints, which must be applied…