Related papers: Compact Neural Networks based on the Multiscale En…
Recent studies suggest that context-aware low-rank approximation is a useful tool for compression and fine-tuning of modern large-scale neural networks. In this type of approximation, a norm is weighted by a matrix of input activations,…
Recent work in Deep Learning has re-imagined the representation of data as functions mapping from a coordinate space to an underlying continuous signal. When such functions are approximated by neural networks this introduces a compelling…
We propose a scalable tensorization framework for neural network compression based on slice-wise feature distillation. Unlike conventional tensor decomposition methods that rely on costly global finetuning, our approach decomposes the…
Deep neural networks currently demonstrate state-of-the-art performance in several domains. At the same time, models of this class are very demanding in terms of computational resources. In particular, a large amount of memory is required…
Deep neural networks have demonstrated state-of-the-art performance in a variety of real-world applications. In order to obtain performance gains, these networks have grown larger and deeper, containing millions or even billions of…
A tensor network is a type of decomposition used to express and approximate large arrays of data. A given data-set, quantum state or higher dimensional multi-linear map is factored and approximated by a composition of smaller multi-linear…
In this paper we review basic and emerging models and associated algorithms for large-scale tensor networks, especially Tensor Train (TT) decompositions using novel mathematical and graphical representations. We discus the concept of…
We propose a new tensor network renormalization group (TNR) scheme based on global optimization and introduce a new method for constructing the finite-temperature density matrix of two-dimensional quantum systems. Combining these two into a…
We consider the problem of deep neural net compression by quantization: given a large, reference net, we want to quantize its real-valued weights using a codebook with $K$ entries so that the training loss of the quantized net is minimal.…
In this work we provide additional support for the proposed connection between the gauge/gravity dualities in string theory and the successful Multi-Scale-Entanglement-Renormalization-anstaz (MERA) method developed for the efficient…
We introduce a coarse-graining transformation for tensor networks that can be applied to study both the partition function of a classical statistical system and the Euclidean path integral of a quantum many-body system. The scheme is based…
In this paper, we present connections between three models used in different research fields: weighted finite automata~(WFA) from formal languages and linguistics, recurrent neural networks used in machine learning, and tensor networks…
Multireference alignment (MRA) refers to the problem of recovering a signal from noisy samples subject to random circular shifts. Expectation--maximization (EM) and variational approaches use statistical modeling to achieve high accuracy at…
Deep learning and especially the use of Deep Neural Networks (DNNs) provides impressive results in various regression and classification tasks. However, to achieve these results, there is a high demand for computing and storing resources.…
In the holographic correspondence of quantum gravity, a global onsite symmetry at the boundary generally translates to a local gauge symmetry in the bulk. We describe one way how the global boundary onsite symmetries can be gauged within…
The development of efficient machine learning models for molecular systems representation is becoming crucial in scientific research. We introduce TensorNet, an innovative O(3)-equivariant message-passing neural network architecture that…
Similarity metrics such as representational similarity analysis (RSA) and centered kernel alignment (CKA) have been used to compare layer-wise representations between neural networks. However, these metrics are confounded by the population…
Advanced tensor decomposition, such as Tensor train (TT) and Tensor ring (TR), has been widely studied for deep neural network (DNN) model compression, especially for recurrent neural networks (RNNs). However, compressing convolutional…
In tensor networks, a geometric operation of pushing a bond cut surface toward a minimal surface corresponds to entanglement distillation. Cutting bonds defines a reduced transition matrix on the bond cut surface and the associated quantum…
In 1+1-dimensional conformal field theory, the thermal state on a circle is related to a certain quotient of the vacuum on a line. We explain how to take this quotient in the MERA tensor network representation of the vacuum and confirm the…