Related papers: Quaternion Backpropagation
The de facto algorithm for training the back pass of a feedforward neural network is backpropagation (BP). The use of almost-everywhere differentiable activation functions made it efficient and effective to propagate the gradient backwards…
Many modern numerical methods in computational science and engineering rely on derivatives of mathematical models for the phenomena under investigation. The computation of these derivatives often represents the bottleneck in terms of…
Earlier versions proposed Graded Projection Recursion (GPR) as a deterministic packed-recursion framework for model-honest near-quadratic dense matrix multiplication. This revised version withdraws the exact dense matrix multiplication…
The success of modern deep learning hinges on the ability to train neural networks at scale. Through clever reuse of intermediate information, backpropagation facilitates training through gradient computation at a total cost roughly…
In this paper, we propose novel quaternion activation functions where we modify either the quaternion magnitude or the phase, as an alternative to the commonly used split activation functions. We define criteria that are relevant for…
In this paper, we show that feedforward and recurrent neural networks exhibit an outer product derivative structure but that convolutional neural networks do not. This structure makes it possible to use higher-order information without…
Randomized algorithms for low-rank approximation of quaternion matrices have gained increasing attention in recent years. However, existing methods overlook pass efficiency, the ability to limit the number of passes over the input…
Using backpropagation to compute gradients of objective functions for optimization has remained a mainstay of machine learning. Backpropagation, or reverse-mode differentiation, is a special case within the general family of automatic…
We derive recursions for the probability distribution of random sums by computer algebra. Unlike the well-known Panjer-type recursions, they are of finite order and thus allow for computation in linear time. This efficiency is bought by the…
We present here a regress later based Monte Carlo approach that uses neural networks for pricing high-dimensional contingent claims. The choice of specific architecture of the neural networks used in the proposed algorithm provides for…
Back-propagation is a popular machine learning algorithm that uses gradient descent in training neural networks for supervised learning, but can be very slow. A number of algorithms have been developed to speed up convergence and improve…
Quaternions are important for a wide variety of rotation-related problems in computer graphics, machine vision, and robotics. We study the nontrivial geometry of the relationship between quaternions and rotation matrices by exploiting the…
The computation of generalized inverses of quaternion matrices is a fundamental problem in quaternion linear algebra, with wide-ranging applications in signal processing, image restoration, and multidimensional data analysis. This paper…
Backpropagation algorithm is indispensable for the training of feedforward neural networks. It requires propagating error gradients sequentially from the output layer all the way back to the input layer. The backward locking in…
This paper proposes a fractional order gradient method for the backward propagation of convolutional neural networks. To overcome the problem that fractional order gradient method cannot converge to real extreme point, a simplified…
The back-propagation algorithm is the cornerstone of deep learning. Despite its importance, few variations of the algorithm have been attempted. This work presents an approach to discover new variations of the back-propagation equation. We…
As an expansion of complex numbers, the quaternions show close relations to numerous physically fundamental concepts. In spite of that, the didactic potential provided by quaternion interrelationships in formulating physical laws are hardly…
The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. A central challenge in this setting is backpropagation through the solution of an…
The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. A central challenge in this setting is backpropagation through the solution of an…
This letter generalizes the Graph Signal Recovery (GSR) problem in Graph Signal Processing (GSP) to the Quaternion domain. It extends the Quaternion Least Mean Square (QLMS) in adaptive filtering literature, and Graph LMS (GLMS) algorithm…