Related papers: Cost optimized ab initio tensor network state meth…
Transformer-based models are becoming deeper and larger recently. For better scalability, an underlying training solution in industry is to split billions of parameters (tensors) into many tasks and then run them across homogeneous…
We propose a new renormalization scheme of tensor networks made only of third order tensors. The isometry used for coarse-graining the network can be prepared at an $O(D^6)$ computational cost in any $d$ dimension ($d \ge 2$), where $D$ is…
Symmetric tensor operations arise in a wide variety of computations. However, the benefits of exploiting symmetry in order to reduce storage and computation is in conflict with a desire to simplify memory access patterns. In this paper, we…
We propose a pair of approximations that allows the leading order computational cost of contracting an infinite projected entangled-pair state (iPEPS) to be reduced from $\mathcal{O}(\chi^3D^6)$ to $\mathcal{O}(\chi^3D^3)$ when using a…
Optimizer states occupy massive GPU memory in large-scale model training. However, gradients in different network blocks exhibit distinct behaviors, such as varying directional stability and scale anisotropy, implying that expensive…
This study aims to solve the over-reliance on the rank estimation strategy in the standard tensor factorization-based tensor recovery and the problem of a large computational cost in the standard t-SVD-based tensor recovery. To this end, we…
We demonstrate the use of tensor networks for image classification with the TensorNetwork open source library. We explain in detail the encoding of image data into a matrix product state form, and describe how to contract the network in a…
Linear recurrent neural networks enable powerful long-range sequence modeling with constant memory usage and time-per-token during inference. These architectures hold promise for streaming applications at the edge, but deployment in…
Classical simulation of quantum computation is necessary for studying the numerical behavior of quantum algorithms, as there does not yet exist a large viable quantum computer on which to perform numerical tests. Tensor network (TN)…
Training large AI models such as LLMs and DLRMs costs massive GPUs and computing time. The high training cost has become only affordable to big tech companies, meanwhile also causing increasing concerns about the environmental impact. This…
While multilinear algebra appears natural for studying the multiway interactions modeled by hypergraphs, tensor methods for general hypergraphs have been stymied by theoretical and practical barriers. A recently proposed adjacency tensor is…
Computing free energy is a fundamental problem in statistical physics. Recently, two distinct methods have been developed and have demonstrated remarkable success: the tensor-network-based contraction method and the neural-network-based…
This paper presents a branch-and-bound algorithm, enhanced with bin packing strategies, for scheduling under variable energy pricing and power-saving states. The proposed algorithm addresses the 1,TOU|states|TEC problem, which involves…
Efficient simulation of quantum circuits has become indispensable with the rapid development of quantum hardware. The primary simulation methods are based on state vectors and tensor networks. As the number of qubits and quantum gates grows…
Nature evolves structures like honeycombs at optimized performance with limited material. These efficient structures can be artificially created with the collaboration of structural topology optimization and additive manufacturing. However,…
We develop a tensor network technique that can solve universal reversible classical computational problems, formulated as vertex models on a square lattice [Nat. Commun. 8, 15303 (2017)]. By encoding the truth table of each vertex…
Tensor networks provide an efficient approximation of operations involving high dimensional tensors and have been extensively used in modelling quantum many-body systems. More recently, supervised learning has been attempted with tensor…
Tensor network techniques are becoming increasingly popular tools to solve partial differential equations within the so-called quantics representation. Their popularity stems from the fact that their spatial resolution depends only…
Sparse matricized tensor times Khatri-Rao product (MTTKRP) is one of the most computationally expensive kernels in sparse tensor computations. This work focuses on optimizing the MTTKRP operation on GPUs, addressing both performance and…
We introduce an efficient algorithm for reducing bond dimensions in an arbitrary tensor network without changing its geometry. The method is based on a novel, quantitative understanding of local correlations in a network. Together with a…