Related papers: Reverse derivative categories
A predilator is a particularly uniform transformation of linear orders. We have a dilator when the transformation preserves well-foundedness. Over the theory $\mathsf{ACA}_0$ from reverse mathematics, any $\Pi^1_2$-formula is equivalent to…
In this note we derive the backward (automatic) differentiation (adjoint [automatic] differentiation) for an algorithm containing a conditional expectation operator. As an example we consider the backward algorithm as it is used in Bermudan…
Let $\md^b(A)$ be the derived category of a finite dimensional basic algebra $A$ with finite global dimension. We construct the Lie algebra arising from the 2-periodic version $\mk_2(\mp(A))$ of $\mk^b(\mp(A))$ in term of constructible…
Consider a normal function $f$ on the ordinals (i. e. a function $f$ that is strictly increasing and continuous at limit stages). By enumerating the fixed points of $f$ we obtain a faster normal function $f'$, called the derivative of $f$.…
We study a class of deep neural networks with networks that form a directed acyclic graph (DAG). For backpropagation defined by gradient descent with adaptive momentum, we show weights converge for a large class of nonlinear activation…
We show how to define forward- and reverse-mode automatic differentiation source-code transformations or on a standard higher-order functional language. The transformations generate purely functional code, and they are principled in the…
If a graph has a non-singular adjacency matrix, then one may use the inverse matrix to define a (labeled) graph that may be considered to be the inverse graph to the original one. It has been known that an adjacency matrix of a tree is…
Moore introduced a class of real-valued "recursive" functions by analogy with Kleene's formulation of the standard recursive functions. While his concise definition inspired a new line of research on analog computation, it contains some…
Containers conveniently represent a wide class of inductive data types. Their derivatives compute representations of types of one-hole contexts, useful for implementing tree-traversal algorithms. In the category of containers and cartesian…
We investigate causal computations taking sequences of inputs to sequences of outputs where the $n$th output depends on the first $n$ inputs only. We model these in category theory via a construction taking a Cartesian category $C$ to…
This manuscript proposes a generalized inverse for a dual matrix called dual Drazin generalized inverse (DDGI) which generalizes the notion of the dual group generalized inverse (DGGI). Under certain necessary and sufficient conditions, we…
Algorithmic differentiation (AD) tools allow to obtain gradient information of a continuously differentiable objective function in a computationally cheap way using the so-called backward mode. It is common practice to use the same tools…
Automatic differentiation, also known as backpropagation, AD, autodiff, or algorithmic differentiation, is a popular technique for computing derivatives of computer programs accurately and efficiently. Sometimes, however, the derivatives…
Markov categories are the central framework for categorical probability theory. Many important concepts from probability theory can be formalized in terms of Markov categories. In particular, conditional probability distributions and Bayes'…
Reverse Mathematics is a program in the foundations of mathematics which provides an elegant classification of theorems of ordinary mathematics based on computability. Our aim is to provide an alternative classification of theorems based on…
This article provides an overview of some of the mathematical principles of Automatic Differentiation (AD). In particular, we summarise different descriptions of the Forward Mode of AD, like the matrix-vector product based approach, the…
The central purpose of this article is to establish new inverse and implicit function theorems for differentiable maps with isolated critical points. One of the key ingredients is a discovery of the fact that differentiable maps with…
I consider differential of mapping $f$ of continuous division ring as linear mapping the most close to mapping $f$. Different expressions which correspond to known deffinition of derivative are supplementary. I explore the Gateaux…
Inverse problems, where in broad sense the task is to learn from the noisy response about some unknown function, usually represented as the argument of some known functional form, has received wide attention in the general scientific…
The problem of inverting a system in presence of a series-defined output is analyzed. Inverse models are derived that consist of a set of algebraic equations. The inversion is performed explicitly for an output trajectory functional, which…