Related papers: Reverse derivative categories
This course, intended for undergraduates familiar with elementary calculus and linear algebra, introduces the extension of differential calculus to functions on more general vector spaces, such as functions that take as input a matrix and…
Automatic differentiation---the mechanical transformation of numeric computer programs to calculate derivatives efficiently and accurately---dates to the origin of the computer age. Reverse mode automatic differentiation both antedates and…
With the increased interest in machine learning, and deep learning in particular, the use of automatic differentiation has become more wide-spread in computation. There have been two recent developments to provide the theoretical support…
Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and other uses of large-scale optimization. Commonly used RAD algorithms such as backpropagation, however, are complex and stateful, hindering deep…
Recent theoretical work on automatic differentiation (autodiff) has focused on characteristics such as correctness and efficiency while assuming that all derivatives are automatically generated by autodiff using program transformation, with…
We define a new perverse t-exact pullback operation on derived categories of constructible sheaves which generalizes most perverse t-exact functors in sheaf theory, such as microlocalization, the Fourier-Sato transform and vanishing cycles.…
Dagger categories are an essential tool for categorical descriptions of quantum physics, for example in categorical quantum mechanics and unitary topological field theory. Their definition however is in tension with the ``principle of…
An inversive meadow is a commutative ring with identity equipped with a multiplicative inverse operation made total by choosing 0 as its value at 0. Previously, inversive meadows were shortly called meadows. A divisive meadow is an…
Differential categories axiomatize the basics of differentiation and provide categorical models of differential linear logic. A differential category is said to have antiderivatives if a natural transformation $\mathsf{K}$, which all…
Reversing a (forward) computation history means undoing the history. In concurrent systems, undoing the history is not performed in a deterministic way but in a causally consistent fashion, where states that are reached during a backward…
Contrastive divergence (CD) learning is a classical method for fitting unnormalized statistical models to data samples. Despite its wide-spread use, the convergence properties of this algorithm are still not well understood. The main source…
Differential categories are now an established abstract setting for differentiation. However not much attention has been given to the process which is inverse to differentiation: integration. This paper presents the parallel development for…
To an inverse semigroup, we associate an \'etale groupoid such that its actions on topological spaces are equivalent to actions of the inverse semigroup. Both the object and the arrow space of this groupoid are non-Hausdorff. We show that…
Cartesian differential categories are categories equipped with a differential combinator which axiomatizes the directional derivative. Important models of Cartesian differential categories include classical differential calculus of smooth…
Automatic differentiation plays a prominent role in scientific computing and in modern machine learning, often in the context of powerful programming systems. The relation of the various embodiments of automatic differentiation to the…
We propose a categorical semantics of gradient-based machine learning algorithms in terms of lenses, parametrised maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework: it…
Backpropagation algorithm is the cornerstone for neural network analysis. Paper extends it for training any derivatives of neural network's output with respect to its input. By the dint of it feedforward networks can be used to solve or…
Within the context of an involutive monoidal category the notion of a comparison relation is identified. Instances are equality on sets, inequality on posets, orthogonality on orthomodular lattices, non-empty intersection on powersets, and…
In differential geometry, the existence of pullbacks is a delicate matter, since the category of smooth manifolds does not admit all of them. When pullbacks are required, often submersions are employed as an ideal class of maps which…
The development of the back-propagation algorithm represents a landmark in neural networks. We provide an approach that conducts the back-propagation again to reverse the traditional back-propagation process to optimize the input loss at…