Related papers: A Hitchhiker's Guide to Automatic Differentiation
We present principles of algebraic diversity (AD), a group-theoretic approach to signal processing exploiting signal symmetry to extract more information per observation, complementing classical methods that use temporal and spatial…
In this work, we discuss the Automatic Adjoint Differentiation (AAD) for functions of the form $G=\frac{1}{2}\sum_1^m (Ey_i-C_i)^2$, which often appear in the calibration of stochastic models. { We demonstrate that it allows a perfect…
Automatic differentiation is a key component in deep learning. This topic is well studied and excellent surveys such as Baydin et al. (2018) have been available to clearly describe the basic concepts. Further, sophisticated implementations…
This document presents a new C++ Automatic Differentiation (AD) tool, AD-HOC (Automatic Differentiation for High-Order Calculations). This tool aims to have the following features: -Calculation of user specified derivatives of arbitrary…
We investigate the automatic differentiation of hybrid models, viz. models that may contain delays, logical tests and discontinuities or loops. We consider differentiation with respect to parameters, initial conditions or the time. We…
Formal transformations somehow resembling the usual derivative are surprisingly common in computer science, with two notable examples being derivatives of regular expressions and derivatives of types. A newcomer to this list is the…
En este trabajo se presenta una propuesta para realizar Diferenciaci\'on Autom\'atica Anidada utilizando cualquier biblioteca de Diferenciaci\'on Autom\'atica que permita sobrecarga de operadores. Para calcular las derivadas anidadas en una…
The application of operator overloading algorithmic differentiation (AD) to computer programs in order to compute the derivative is quite common. But, the replacement of the underlying computational floating point type with the specialized…
We present a novel framework, namely AADMM, for acceleration of linearized alternating direction method of multipliers (ADMM). The basic idea of AADMM is to incorporate a multi-step acceleration scheme into linearized ADMM. We demonstrate…
An activation function is an element-wise mathematical function and plays a crucial role in deep neural networks (DNN). Many novel and sophisticated activation functions have been proposed to improve the DNN accuracy but also consume…
Various software efforts embrace the idea that object oriented programming enables a convenient implementation of the chain rule, facilitating so-called automatic differentiation via backpropagation. Such frameworks have no mechanism for…
In practice, machine learning methods commonly require anomaly detection (AD) to filter inputs or detect distributional shifts. Typically, this is implemented by running a separate AD model alongside the primary model. However, this…
Automatic differentiation is a technique which allows a programmer to define a numerical computation via compositions of a broad range of numeric and computational primitives and have the underlying system support the computation of partial…
We present a technique for applying (forward and) reverse-mode automatic differentiation (AD) on a non-recursive second-order functional array language that supports nested parallelism and is primarily aimed at efficient GPU execution. The…
En este trabajo se presenta una propuesta para realizar Diferenciaci\'on Autom\'atica Anidada utilizando cualquier biblioteca de Diferenciaci\'on Autom\'atica que permita sobrecarga de operadores. Para calcular las derivadas anidadas en una…
The aspiration of the next generation's autonomous driving (AD) technology relies on the dedicated integration and interaction among intelligent perception, prediction, planning, and low-level control. There has been a huge bottleneck…
Derivative-based algorithms are ubiquitous in statistics, machine learning, and applied mathematics. Automatic differentiation offers an algorithmic way to efficiently evaluate these derivatives from computer programs that execute relevant…
This article aims to demonstrate and discuss the applications of automatic differentiation (AD) for finding derivatives in PDE-constrained optimization problems and Jacobians in non-linear finite element analysis. The main idea is to…
The extraction of the model parameters is as important as the development of compact model itself because simulation accuracy is fully determined by the accuracy of the parameters used. This study proposes an efficient model-parameter…
Adaptive optimizers, such as Adam, have achieved remarkable success in deep learning. A key component of these optimizers is the so-called preconditioning matrix, providing enhanced gradient information and regulating the step size of each…