Related papers: LC-SVD-DLinear: A low-cost physics-based hybrid ma…
Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones. Existing methods typically rely on low-rank,…
The deployment of Large Language Models is constrained by the memory and bandwidth demands of static weights and dynamic Key-Value cache. SVD-based compression provides a hardware-friendly solution to reduce these costs. However, existing…
The singular value decomposition (SVD) is not only a classical theory in matrix computation and analysis, but also is a powerful tool in machine learning and modern data analysis. In this tutorial we first study the basic notion of SVD and…
The generalized singular value decomposition (GSVD) is a valuable tool that has many applications in computational science. However, computing the GSVD for large-scale problems is challenging. Motivated by applications in hyper-differential…
We propose an efficient, distributed, out-of-memory implementation of the truncated singular value decomposition (t-SVD) for heterogeneous (CPU+GPU) high performance computing (HPC) systems. Various implementations of SVD have been…
The traditional method of computing singular value decomposition (SVD) of a data matrix is based on a least squares principle, thus, is very sensitive to the presence of outliers. Hence the resulting inferences across different applications…
The advancements in Large Language Models (LLMs) have been hindered by their substantial sizes, which necessitates LLM compression methods for practical deployment. Singular Value Decomposition (SVD) offers a promising solution for LLM…
Singular value decomposition (SVD) is one of the most popular compression methods that approximate a target matrix with smaller matrices. However, standard SVD treats the parameters within the matrix with equal importance, which is a simple…
This paper presents the modal truncation and singular value decomposition (SVD) technique as two main algorithms for dynamic model reduction of the power system. The significance and accuracy of the proposed methods are investigated with…
A novel data-driven method of modal analysis for complex flow dynamics, termed as reduced-order variational mode decomposition (RVMD), has been proposed, combining the idea of the separation of variables and a state-of-the-art nonstationary…
By singular value decomposition (SVD) of a numerically singular Hessian matrix and a numerically singular system of linear equations for the experimental data (accumulated in the respective ${\chi ^2}$ function) and constraints, least…
The randomized singular value decomposition (R-SVD) is a popular sketching-based algorithm for efficiently computing the partial SVD of a large matrix. When the matrix is low-rank, the R-SVD produces its partial SVD exactly; but when the…
Streamlined weirs which are a nature-inspired type of weir have gained tremendous attention among hydraulic engineers, mainly owing to their established performance with high discharge coefficients. Computational fluid dynamics (CFD) is…
Analyzing complex experimental data with multiple parameters is challenging. We propose using Singular Value Decomposition (SVD) as an effective solution. This method, demonstrated through real experimental data analysis, surpasses…
In this letter, we propose a simple yet effective singular value decomposition (SVD) based strategy to reduce the optimization problem dimension in data-enabled predictive control (DeePC). Specifically, in the case of linear time-invariant…
Singular Value Decomposition (SVD) is one of the most useful techniques for analyzing data in linear algebra. SVD decomposes a rectangular real or complex matrix into two orthogonal matrices and one diagonal matrix. In this work we…
We present a simple yet novel parameterized form of linear mapping to achieves remarkable network compression performance: a pseudo SVD called Ternary SVD (TSVD). Unlike vanilla SVD, TSVD limits the $U$ and $V$ matrices in SVD to ternary…
Very deep convolutional neural networks (CNNs) have been firmly established as the primary methods for many computer vision tasks. However, most state-of-the-art CNNs are large, which results in high inference latency. Recently, depth-wise…
Linear transformation techniques such as singular value decomposition (SVD) have been used widely to gain insight into the qualitative dynamics of data generated by dynamical systems. There have been several reports in the past that had…
The reconstruction and prediction of full-state flows from sparse data are of great scientific and engineering significance yet remain challenging, especially in applications where data are sparse and/or subjected to noise. To this end,…