Related papers: A Geometric Theory of Higher-Order Automatic Diffe…
Scientific studies often require the precise calculation of derivatives. In many cases an analytical calculation is not feasible and one resorts to evaluating derivatives numerically. These are error-prone, especially for higher-order…
In this paper we take a look at Automatic Differentiation through the eyes of Tensor and Operational Calculus. This work is best consumed as supplementary material for learning tensor and operational calculus by those already familiar with…
The nature of so-called differential-algebraic operators and their approximations is constitutive for the direct treatment of higher-index differential-algebraic equations. We treat first-order differential-algebraic operators in detail and…
Derivatives play a critical role in computational statistics, examples being Bayesian inference using Hamiltonian Monte Carlo sampling and the training of neural networks. Automatic differentiation is a powerful tool to automate the…
We propose an algebraic geometric approach for studying rational solutions of first-order algebraic ordinary difference equations. For an autonomous first-order algebraic ordinary difference equations, we give an upper bound for the degrees…
In scientific computation, it is often necessary to calculate higher-order derivatives of a function. Currently, two primary methods for higher-order automatic differentiation exist: symbolic differentiation and algorithmic automatic…
Higher order derivatives of functions are structured high dimensional objects which lend themselves to many alternative representations, with the most popular being multi-index, matrix and tensor representations. The choice between them…
First-order operator splitting methods are ubiquitous among many fields through science and engineering, such as inverse problems, signal/image processing, statistics, data science and machine learning, to name a few. In this paper, we…
Automatic differentiation is a key component in deep learning. This topic is well studied and excellent surveys such as Baydin et al. (2018) have been available to clearly describe the basic concepts. Further, sophisticated implementations…
Automatic differentiation---the mechanical transformation of numeric computer programs to calculate derivatives efficiently and accurately---dates to the origin of the computer age. Reverse mode automatic differentiation both antedates and…
This paper presents an algebraic approach to characterizing higher-order differential operators. While the foundational Leibniz rule addresses first-order derivatives, its extension to higher orders typically involves identities relating…
Parameter choosing in classical edge detection algorithms can be a costly and complex task. Choosing the correct parameters can improve considerably the resulting edge-map. In this paper we present a version of Edge Drawing algorithm in…
We present a novel approach for high-order accurate numerical differentiation on unstructured meshes of quadrilateral elements. To differentiate a given function, an auxiliary function with greater smoothness properties is defined which…
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD), also called algorithmic differentiation or simply "autodiff", is a family of techniques similar to but more…
A computational revolution unleashed the power of artificial neural networks. At the heart of that revolution is automatic differentiation, which calculates the derivative of a performance measure relative to a large number of parameters.…
In the past decades, the finite difference methods for space fractional operators develop rapidly; to the best of our knowledge, all the existing finite difference schemes, including the first and high order ones, just work on uniform…
Recent theoretical work on automatic differentiation (autodiff) has focused on characteristics such as correctness and efficiency while assuming that all derivatives are automatically generated by autodiff using program transformation, with…
We introduce basic aspects of new operator method, which is very suitable for practical solving differential equations of various types. The main advantage of the method is revealed in opportunity to find compact exact operator solutions of…
A model of a geometric algorithm is introduced and methodology of its operation is presented for the dynamic partitioning of data spaces.
Based on a class of associative algebras with zero-divisors which are called real-like algebras by us, we introduce a way of defining automatic differentiation and present different ways of doing automatic differentiation to compute the…