Related papers: Compression Properties for large Toeplitz-like mat…
Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which…
We present a fast direct solver for structured linear systems based on multilevel matrix compression. Using the recently developed interpolative decomposition of a low-rank matrix in a recursive manner, we embed an approximation of the…
A key question in many low-rank problems throughout optimization, machine learning, and statistics is to characterize the convex hulls of simple low-rank sets and judiciously apply these convex hulls to obtain strong yet computationally…
Deep neural networks have achieved strong performance in image classification tasks due to their ability to learn complex patterns from high-dimensional data. However, their large computational and memory requirements often limit deployment…
Nonnegative matrix factorization (NMF) has an established reputation as a useful data analysis technique in numerous applications. However, its usage in practical situations is undergoing challenges in recent years. The fundamental factor…
How can we compress language models without sacrificing accuracy? The number of compression algorithms for language models is rapidly growing to benefit from remarkable advances of recent language models without side effects due to the…
The Hadamard decomposition is a powerful technique for data analysis and matrix compression, which decomposes a given matrix into the element-wise product of two or more low-rank matrices. In this paper, we develop an efficient algorithm to…
Folding grid value vectors of size $2^L$ into $L$th order tensors of mode sizes $2\times \cdots\times 2$, combined with low-rank representation in the tensor train format, has been shown to lead to highly efficient approximations for…
The embedding layers transforming input words into real vectors are the key components of deep neural networks used in natural language processing. However, when the vocabulary is large, the corresponding weight matrices can be enormous,…
Tensor decomposition is one of the fundamental technique for model compression of deep convolution neural networks owing to its ability to reveal the latent relations among complex structures. However, most existing methods compress the…
The main focus of this paper is the study of efficient multigrid methods for large linear systems with a particular saddle-point structure. Indeed, when the system matrix is symmetric, but indefinite, the variational convergence theory that…
We describe a simple, black-box compression format for tensors with a multiscale structure. By representing the tensor as a sum of compressed tensors defined on increasingly coarse grids, we capture low-rank structures on each grid-scale,…
The prohibitive sizes of Large Language Models (LLMs) today make it difficult to deploy them on memory-constrained edge devices. This work introduces $\rm CALDERA$ -- a new post-training LLM compression algorithm that harnesses the inherent…
We introduce an adaptive structured low rank algorithm to recover MR images from their undersampled Fourier coefficients. The image is modeled as a combination of a piecewise constant component and a piecewise linear component. The Fourier…
We show that every n-by-n matrix is generically a product of [n/2] + 1 Toeplitz matrices and always a product of at most 2n+5 Toeplitz matrices. The same result holds true if the word "Toeplitz" is replaced by "Hankel", and the generic…
Deep Convolutional Neural Networks (CNNs) are increasingly difficult to deploy on microcontrollers (MCUs) and lightweight NPUs (Neural Processing Units) due to their growing size and compute demands. Low-rank tensor decomposition, such as…
We are interested in solving linear systems arising from three applications: (1) kernel methods in machine learning, (2) discretization of boundary integral equations from mathematical physics, and (3) Schur complements formed in the…
The recently developed theory of higher--rank numerical ranges originated in problems of error correction in quantum information theory but its mathematical implications now include a quite satisfactory understanding of \emph{scalar}…
The stories told in this paper are dealing with the solution of finite, infinite, and biinfinite Toeplitz-type systems. A crucial role plays the off-diagonal decay behavior of Toeplitz matrices and their inverses. Classical results of…
We present complete characterizations of Toeplitz operators that are complex symmetric. This follows as a by-product of characterizations of conjugations on Hilbert spaces. Notably, we prove that every conjugation admits a canonical…