Related papers: Matrix Calculus (for Machine Learning and Beyond)
Fractional gradient descent has been studied extensively, with a focus on its ability to extend traditional gradient descent methods by incorporating fractional-order derivatives. This approach allows for more flexibility in navigating…
Optimization models involving quaternion matrices are widely used in color image process and other engineering areas. These models optimize real functions of quaternion matrix variables. In particular, $\ell_0$-norms and rank functions of…
Differential machine learning combines automatic adjoint differentiation (AAD) with modern machine learning (ML) in the context of risk management of financial Derivatives. We introduce novel algorithms for training fast, accurate pricing…
The computation of matrix functions is a well-studied problem. Of special importance are the exponential and the logarithm of a matrix, where the latter also raises existence and uniqueness questions. This is particularly relevant in the…
The article presents a matrix differential operator and a pseudoinverse matrix differential operator for finding a particular solution to nonhomogeneous linear ordinary differential equations (ODE) with constant coefficients with special…
An efficient coordinate-free notation is elucidated for differentiating matrix expressions and other functions between higher-dimensional vector spaces. This method of differentiation is known, but not explained well, in the literature.…
These lecture notes provide a self-contained introduction to the mathematical methods required in a Bachelor degree programme in Business, Economics, or Management. In particular, the topics covered comprise real-valued vector and matrix…
Factorization machines and polynomial networks are supervised polynomial models based on an efficient low-rank decomposition. We extend these models to the multi-output setting, i.e., for learning vector-valued functions, with application…
In calculating integral or discrete transforms, use has been made of fast algorithms for multiplying vectors by matrices whose elements are specified as values of special (Chebyshev, Legendre, Laguerre, etc.) functions. The currently…
Linear algebraic expressions are the essence of many computationally intensive problems, including scientific simulations and machine learning applications. However, translating high-level formulations of these expressions to efficient…
The goal of this thesis is threefold: first, to provide a general semantic setting for reasoning about incremental computation. Second, to establish and clarify the connection between derivatives in the incremental sense and derivatives in…
Model Agnostic Meta Learning (MAML) is widely used to find a good initialization for a family of tasks. Despite its success, a critical challenge in MAML is to calculate the gradient w.r.t. the initialization of a long training trajectory…
A variety of problems emerged investigating electronic circuits, computer devices and cellular automata motivated a number of attempts to create a differential and integral calculus for Boolean functions. In the present article, we extend…
One of the grand challenges of Mathematics instruction is to provide students with problems that are both accessible and have a reasonably elegant solution. Instructors commonly resort to resources like course textbooks, online-learning…
Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and other uses of large-scale optimization. Commonly used RAD algorithms such as backpropagation, however, are complex and stateful, hindering deep…
This article provides an overview of some of the mathematical principles of Automatic Differentiation (AD). In particular, we summarise different descriptions of the Forward Mode of AD, like the matrix-vector product based approach, the…
The scope of this teaching package is to make a brief induction to Artificial Neural Networks (ANNs) for people who have no previous knowledge of them. We first make a brief introduction to models of networks, for then describing in general…
This is a set of lecture notes for a graduate-level course on quantum algorithms, with an emphasis on quantum optimization algorithms. It is developed for applied mathematicians and engineers, and requires no previous background in quantum…
This short course offers a new perspective on randomized algorithms for matrix computations. It explores the distinct ways in which probability can be used to design algorithms for numerical linear algebra. Each design template is…
Causal functions of sequences occur throughout computer science, from theory to hardware to machine learning. Mealy machines, synchronous digital circuits, signal flow graphs, and recurrent neural networks all have behaviour that can be…