相关论文: Automatic Differentiation: Theory and Practice

It is well known that the action functional can be used to define classical, quantum, closed, and open dynamics in a generalization of the variational principle and in the path integral formalism in classical and quantum dynamics,…

Automatic differentiation (AD) is a technique for computing the derivative of a function represented by a program. This technique is considered as the de-facto standard for computing the differentiation in many machine learning and…

Differentiable programming has facilitated numerous methodological advances in scientific computing. Physics engines supporting automatic differentiation have simpler code, accelerating the development process and reducing the maintenance…

Building on the observation that reverse-mode automatic differentiation (AD) -- a generalisation of backpropagation -- can naturally be expressed as pullbacks of differential 1-forms, we design a simple higher-order programming language…

Automatic differentiation plays a prominent role in scientific computing and in modern machine learning, often in the context of powerful programming systems. The relation of the various embodiments of automatic differentiation to the…

Automatic differentiation (AD) is a range of algorithms to compute the numeric value of a function's (partial) derivative, where the function is typically given as a computer program or abstract syntax tree. AD has become immensely popular…

This paper provides a new approach to derive various arbitrary high order finite difference formulae for the numerical differentiation of analytic functions. In this approach, various first and second order formulae for the numerical…

We present semantic correctness proofs of automatic differentiation (AD). We consider a forward-mode AD method on a higher-order language with algebraic data types and we characterise it as the unique structure-preserving macro given a…

A large class of non-smooth practical optimization problems can be written as minimization of a sum of smooth and partly smooth functions. We examine such structured problems which also depend on a parameter vector and study the problem of…

The issue of non-perturbative background independent quantization of matrix models is addressed. The analysis is carried out by considering a simple matrix model which is a matrix extension of ordinary mechanics reduced to 0 dimension. It…

Reverse-mode differentiation is used for optimization, but it introduces references, which break the purity of the underlying programs, making them notoriously harder to optimize. We present a reverse-mode differentiation on a purely…

This chapter describes modal decompositions in the framework of matrix factorizations. We highlight the differences between classic space-time decompositions and 2D discrete transforms and discuss the general architecture underpinning…

For discrete-time systems, flatness is usually defined by replacing the time-derivatives of the well-known continuous-time definition by forward-shifts. With this definition, the class of flat systems corresponds exactly to the class of…

We consider backward filtrations generated by processes coming from deterministic and probabilistic cellular automata. We prove that these filtrations are standard in the classical sense of Vershik's theory, but we also study them from…

We show how to apply forward and reverse mode Combinatory Homomorphic Automatic Differentiation (CHAD) to total functional programming languages with expressive type systems featuring the combination of - tuple types; - sum types; -…

It is shown how to extend the formal variational calculus in order to incorporate integrals of divergences into it. Such a generalization permits to study nontrivial boundary problems in field theory on the base of canonical formalism.

We present a simple functional programming language, called Dual PCF, that implements forward mode automatic differentiation using dual numbers in the framework of exact real number computation. The main new feature of this language is the…

We present a simple technique for semantic, open logical relations arguments about languages with recursive types, which, as we show, follows from a principled foundation in categorical semantics. We demonstrate how it can be used to give a…

This paper presents a simple decidable logic of functional dependence LFD, based on an extension of classical propositional logic with dependence atoms plus dependence quantifiers treated as modalities, within the setting of generalized…

Automatic differentiation is a key component in deep learning. This topic is well studied and excellent surveys such as Baydin et al. (2018) have been available to clearly describe the basic concepts. Further, sophisticated implementations…