Related papers: Automatic Differentiation for Tensor Algebras
Tensor algebra is widely used in many applications, such as scientific computing, machine learning, and data analytics. The tensors represented real-world data are usually large and sparse. There are tens of storage formats designed for…
We introduce and extend the outer product and contractive product of tensors and matrices, and present some identities in terms of these products. We offer tensor expressions of derivatives of tensors, focus on the tensor forms of…
We introduce DISTAL, a compiler for dense tensor algebra that targets modern distributed and heterogeneous systems. DISTAL lets users independently describe how tensors and computation map onto target machines through separate format and…
Computing derivatives of tensor expressions, also known as tensor calculus, is a fundamental task in machine learning. A key concern is the efficiency of evaluating the expressions and their derivatives that hinges on the representation of…
Tensors or {\em multi-way arrays} are functions of three or more indices $(i,j,k,\cdots)$ -- similar to matrices (two-way arrays), which are functions of two indices $(r,c)$ for (row,column). Tensors have a rich history, stretching over…
In this paper we take a look at Automatic Differentiation through the eyes of Tensor and Operational Calculus. This work is best consumed as supplementary material for learning tensor and operational calculus by those already familiar with…
How does one compile derivatives of tensor programs, such that the resulting code is purely functional (hence easier to optimize and parallelize) and provably efficient relative to the original program? We show that naively differentiating…
This document describes an attempt to develop a compiler-based approach for computations with symmetric tensors. Given a computation and the symmetries of its input tensors, we derive formulas for random access under a storage scheme that…
The norm of the $m$th derivative of the map that takes an operator to its $k$th antisymmetric tensor power is evaluated. The case $m=1$ has been studied earlier by Bhatia and Friedland [R. Bhatia and S. Friedland, Variation of Grassman…
This chapter studies the problem of decomposing a tensor into a sum of constituent rank one tensors. While tensor decompositions are very useful in designing learning algorithms and data analysis, they are NP-hard in the worst-case. We will…
We provide formulas for computing the discriminant of noncommutative algebras over central subalgebras in the case of Ore extensions and skew group extensions. The formulas follow from a more general result regarding the discriminants of…
In this paper, we present a partial survey of the tools borrowed from tensor algebra, which have been utilized recently in Statistics and Signal Processing. It is shown why the decompositions well known in linear algebra can hardly be…
Sparse tensors are prevalent in many data-intensive applications, yet existing differentiable programming frameworks are tailored towards dense tensors. This presents a significant challenge for efficiently computing gradients through…
We present new results on the classical algorithm of variable elimination, which underlies many algorithms including for probabilistic inference. The results relate to exploiting functional dependencies, allowing one to perform inference…
Deploying deep learning models on various devices has become an important topic. The wave of hardware specialization brings a diverse set of acceleration primitives for multi-dimensional tensor computations. These new acceleration…
In this paper, based on the combination of tensor neural network and a posteriori error estimator, a novel type of machine learning method is proposed to solve high-dimensional boundary value problems with homogeneous and non-homogeneous…
One of the main issues in computing a tensor decomposition is how to choose the number of rank-one components, since there is no finite algorithms for determining the rank of a tensor. A commonly used approach for this purpose is to find a…
The study of derivations and their generalizations on non-associative algebras has proven to be fundamental in understanding the internal symmetries and algebraic dynamics of such structures. In this paper, we investigate derivations and…
The tensor t-function, a formalism that generalizes the well-known concept of matrix functions to third-order tensors, is introduced in [K. Lund, The tensor t-function: a definition for functions of third-order tensors, Numer. Linear…
Vector algebra is a powerful and needful tool for Physics but unfortunately, due to lack of mathematical skills, it becomes misleading for first undergraduate courses of science and engineering studies. Standard vector identities are…