Related papers: Automatic Differentiation for Tensor Algebras
Following the recent preprints arXiv:1903.09650 and arXiv:1906.04654 we comment on the feasibility of implementation of autodifferentiation in standard tensor network toolkits by briefly walking through the steps to do so. The total…
Tensor algebra is a crucial component for data-intensive workloads such as machine learning and scientific computing. As the complexity of data grows, scientists often encounter a dilemma between the highly specialized dense tensor algebra…
Sparse tensors arise in problems in science, engineering, machine learning, and data analytics. Programs that operate on such tensors can exploit sparsity to reduce storage requirements and computational time. Developing and maintaining…
Many artificial intelligence models process input data of different lengths and resolutions, making the shape of the tensors dynamic. The performance of these models depends on the shape of the tensors, which makes it difficult to optimize…
Feature learning forms the cornerstone for tackling challenging learning problems in domains such as speech, computer vision and natural language processing. In this paper, we consider a novel class of matrix and tensor-valued features,…
We propose the tensor Kronecker product singular value decomposition~(TKPSVD) that decomposes a real $k$-way tensor $\mathcal{A}$ into a linear combination of tensor Kronecker products with an arbitrary number of $d$ factors $\mathcal{A} =…
In this article we will analyse how to compute the contribution of each input value to its aggregate output in some nonlinear models. Regression and classification applications, together with related algorithms for deep neural networks are…
Tensor decomposition is a well-known tool for multiway data analysis. This work proposes using stochastic gradients for efficient generalized canonical polyadic (GCP) tensor decomposition of large-scale tensors. GCP tensor decomposition is…
Derivatives play a critical role in computational statistics, examples being Bayesian inference using Hamiltonian Monte Carlo sampling and the training of neural networks. Automatic differentiation is a powerful tool to automate the…
Recent AI applications such as Collaborative Intelligence with neural networks involve transferring deep feature tensors between various computing devices. This necessitates tensor compression in order to optimize the usage of…
The problem of finding independent components of an indexed object (e.g., a tensor) with arbitrary number of indices and arbitrary linear symmetries is discussed. It is proved that the number of independent components $f(k)$ is a polynomial…
We consider the problem of decomposing a real-valued symmetric tensor as the sum of outer products of real-valued vectors. Algebraic methods exist for computing complex-valued decompositions of symmetric tensors, but here we focus on…
A finite semifield is a division algebra over a finite field where multiplication is not necessarily associative. We consider here the complexity of the multiplication in small semifields and finite field extensions. For this operation, the…
Many problems in high-dimensional statistics appear to have a statistical-computational gap: a range of values of the signal-to-noise ratio where inference is information-theoretically possible, but (conjecturally) computationally…
Suppose we are given an $n$-dimensional order-3 symmetric tensor $T \in (\mathbb{R}^n)^{\otimes 3}$ that is the sum of $r$ random rank-1 terms. The problem of recovering the rank-1 components is possible in principle when $r \lesssim n^2$…
A numerical algorithm to decompose an exact low-rank skew-symmetric tensor into a sum of elementary (rank-$1$) skew-symmetric tensors is introduced. The algorithm uncovers this Grassmann decomposition based on linear relations that are…
Tensor decomposition is an important technique for capturing the high-order interactions among multiway data. Multi-linear tensor composition methods, such as the Tucker decomposition and the CANDECOMP/PARAFAC (CP), assume that the complex…
Canonical Polyadic Decomposition (CPD) of a third-order tensor is a minimal decomposition into a sum of rank-$1$ tensors. We find new mild deterministic conditions for the uniqueness of individual rank-$1$ tensors in CPD and present an…
In this note, we propose a framework for proving computational lower bounds in norm approximation by leveraging a reverse detection--estimation gap. The starting point is a testing problem together with an estimator whose error is…
The theory and computation of tensors with different tensor products play increasingly important roles in scientific computing and machine learning. Different products aim to preserve different algebraic properties from the matrix algebra,…