Related papers: Differential Calculus, Tensor Products and the Imp…
Computing derivatives of tensor expressions, also known as tensor calculus, is a fundamental task in machine learning. A key concern is the efficiency of evaluating the expressions and their derivatives that hinges on the representation of…
We propose a diagrammatic notation for matrix differentiation. Our new notation enables us to derive formulas for matrix differentiation more easily than the usual matrix (or index) notation. We demonstrate the effectiveness of our notation…
Computing multivariate derivatives of matrix-like expressions in the compact, coordinate free fashion is very important for both theory and applied computations (e.g. optimization and machine learning). The critical components of such…
Learning vector calculus techniques is one of the major missions to be accomplished by physics undergraduates. However, beginners report various difficulties dealing with the index notation due to its bulkiness. Meanwhile, there have been…
Tensor network diagram (graphical notation) is a useful tool that graphically represents multiplications between multiple tensors using nodes and edges. Using the graphical notation, complex multiplications between tensors can be described…
Vector algebra is a powerful and needful tool for Physics but unfortunately, due to lack of mathematical skills, it becomes misleading for first undergraduate courses of science and engineering studies. Standard vector identities are…
We present the basic concepts of tensor products of vector spaces, emphasizing linear algebraic and combinatorial techniques as needed for applied areas of research. The topics include (1) Introduction; (2) Basic multilinear algebra; (3)…
In this paper we take a look at Automatic Differentiation through the eyes of Tensor and Operational Calculus. This work is best consumed as supplementary material for learning tensor and operational calculus by those already familiar with…
This course, intended for undergraduates familiar with elementary calculus and linear algebra, introduces the extension of differential calculus to functions on more general vector spaces, such as functions that take as input a matrix and…
Feature learning forms the cornerstone for tackling challenging learning problems in domains such as speech, computer vision and natural language processing. In this paper, we consider a novel class of matrix and tensor-valued features,…
Higher order derivatives of functions are structured high dimensional objects which lend themselves to many alternative representations, with the most popular being multi-index, matrix and tensor representations. The choice between them…
We present a fit-for-purpose introduction to tensors and their operations. It is envisaged to help the reader become acquainted with its underpinning concepts for the study of path signatures. The text includes exercises, solutions and many…
Differentiation is a cornerstone of computing and data analysis in every discipline of science and engineering. Indeed, most fundamental physics laws are expressed as relationships between derivatives in space and time. However, derivatives…
Many critical EDA problems suffer from the curse of dimensionality, i.e. the very fast-scaling computational burden produced by large number of parameters and/or unknown variables. This phenomenon may be caused by multiple spatial or…
Vector calculus in three-dimensional space is ubiquitous in applications of mathematics in physics and engineering. Its two-dimensional version is, however, quite rare. Here we try to provide a pedagogical account of the subject. It is…
Tensor methods have become a promising tool to solve high-dimensional problems in the big data era. By exploiting possible low-rank tensor factorization, many high-dimensional model-based or data-driven problems can be solved to facilitate…
Feature learning forms the cornerstone for tackling challenging learning problems in domains such as speech, computer vision and natural language processing. In this paper, we consider a novel class of matrix and tensor-valued features,…
Complicated mathematical equations involving products of tensors with permutation symmetries, frequently encountered in fields such as general relativity and quantum chemistry (e.g., equations in high-order coupled cluster theories),…
There have been several modifications of how basic calculus has been taught, but very few of these modifications have considered the computational tools available at our disposal. Here, we present a few tools that are easy to develop and…
High-dimensional data arise naturally in many areas of science and engineering, including machine learning, signal processing, computational physics, and statistics. Such data are often represented as tensors, multi-dimensional…