Reverse derivative categories
Logic in Computer Science
2019-10-17 v1 Category Theory
Abstract
The reverse derivative is a fundamental operation in machine learning and automatic differentiation. This paper gives a direct axiomatization of a category with a reverse derivative operation, in a similar style to that given by Cartesian differential categories for a forward derivative. Intriguingly, a category with a reverse derivative also has a forward derivative, but the converse is not true. In fact, we show explicitly what a forward derivative is missing: a reverse derivative is equivalent to a forward derivative with a dagger structure on its subcategory of linear maps. Furthermore, we show that these linear maps form an additively enriched category with dagger biproducts.
Cite
@article{arxiv.1910.07065,
title = {Reverse derivative categories},
author = {Robin Cockett and Geoffrey Cruttwell and Jonathan Gallagher and Jean-Simon Pacaud Lemay and Benjamin MacAdam and Gordon Plotkin and Dorette Pronk},
journal= {arXiv preprint arXiv:1910.07065},
year = {2019}
}
Comments
Extended version of paper to appear at CSL 2020