Related papers: Differential Calculus, Tensor Products and the Imp…
Scientific studies often require the precise calculation of derivatives. In many cases an analytical calculation is not feasible and one resorts to evaluating derivatives numerically. These are error-prone, especially for higher-order…
Tensors, or multi-linear forms, are important objects in a variety of areas from analytics, to combinatorics, to computational complexity theory. Notions of tensor rank aim to quantify the "complexity" of these forms, and are thus also…
These notes are the second part of the tensor calculus documents which started with the previous set of introductory. In the present text, we continue the discussion of selected topics of the subject at a higher level expanding, when…
We state some elementary problems concerning the relation between difference calculus and differential calculus, and we try to convince the reader that, in spite of the simplicity of the statements, a solution of these problems would be a…
Dimensionality reduction is an effective method for learning high-dimensional data, which can provide better understanding of decision boundaries in human-readable low-dimensional subspace. Linear methods, such as principal component…
In this paper the analogy between differential forms arising from integrals in additive calculus and forms arising from the integrals in product calculus is investigated. It is found that with an appropriate definition of scalar…
Automatic differentiation plays a prominent role in scientific computing and in modern machine learning, often in the context of powerful programming systems. The relation of the various embodiments of automatic differentiation to the…
We show how the numerical range of a matrix can be used to bound the optimal value of certain optimization problems over real tensor product vectors. Our bound is stronger than the trivial bounds based on eigenvalues, and can be computed…
Tensors are multidimensional arrays of numerical values and therefore generalize matrices to multiple dimensions. While tensors first emerged in the psychometrics community in the $20^{\text{th}}$ century, they have since then spread to…
Kernel matrices are crucial in many learning tasks such as support vector machines or kernel ridge regression. The kernel matrix is typically dense and large-scale. Depending on the dimension of the feature space even the computation of all…
Derivatives play a critical role in computational statistics, examples being Bayesian inference using Hamiltonian Monte Carlo sampling and the training of neural networks. Automatic differentiation is a powerful tool to automate the…
In mathematics, many notations have been invented for the concise representation of mathematical formulae. Tensor index notation is one of such notations and has been playing a crucial role in describing formulae in mathematical physics.…
We introduce and extend the outer product and contractive product of tensors and matrices, and present some identities in terms of these products. We offer tensor expressions of derivatives of tensors, focus on the tensor forms of…
A tensor in applied mathematics is usually defined as a multidimensional array of numbers. This presumes a choice of basis in $\mathbb{R}^n$ or in some other vector space, and tensorial concepts are defined accordingly. In this article we…
This work concerns the construction and characterization of product kernels for multivariate approximation from a finite set of discrete samples. To this end, we consider composing different component kernels, each acting on a…
The method of choice to study one-dimensional strongly interacting many body quantum systems is based on matrix product states and operators. Such method allows to explore the most relevant, and numerically manageable, portion of an…
The simple product formulae for derivatives of scalar functions raised to different powers are generalized for functions which take values in the set of symmetric positive definite matrices. These formulae are fundamental in derivation of…
There is a significant expansion in both volume and range of applications along with the concomitant increase in the variety of data sources. These ever-expanding trends have highlighted the necessity for more versatile analysis tools that…
We introduce a novel latent vector space model that jointly learns the latent representations of words, e-commerce products and a mapping between the two without the need for explicit annotations. The power of the model lies in its ability…
The proper handling of 3D orientations is a central element in many optimization problems in engineering. Unfortunately many researchers and engineers struggle with the formulation of such problems and often fall back to suboptimal…