Related papers: An adaptive dynamical low-rank optimizer for solvi…
We develop two new sets of stable, rank-adaptive Dynamically Orthogonal Runge-Kutta (DORK) schemes that capture the high-order curvature of the nonlinear low-rank manifold. The DORK schemes asymptotically approximate the truncated singular…
Neural networks have achieved tremendous success in a large variety of applications. However, their memory footprint and computational demand can render them impractical in application settings with limited hardware or energy resources. In…
The stochastic description of chemical reaction networks with the kinetic chemical master equation (CME) is important for studying biological cells, but it suffers from the curse of dimensionality: The amount of data to be stored grows…
This paper proposes an effective low-rank alternating direction doubling algorithm (R-ADDA) for computing numerical low-rank solutions to large-scale sparse continuous-time algebraic Riccati matrix equations. The method is based on the…
Large-scale dynamic inverse problems are often ill-posed due to model complexity and the high dimensionality of the unknown parameters. Regularization is commonly employed to mitigate ill-posedness by incorporating prior information and…
Projected gradient descent and its Riemannian variant belong to a typical class of methods for low-rank matrix estimation. This paper proposes a new Nesterov's Accelerated Riemannian Gradient algorithm by efficient orthographic retraction…
We introduce a reformulation of regularized low-rank recovery models to take advantage of GPU, multiple CPU, and hybridized architectures. Low-rank recovery often involves nuclear-norm minimization through iterative thresholding of singular…
This paper proposes a reduced-rank scheme for adaptive beamforming based on the constrained joint iterative optimization of filters. We employ this scheme to devise two novel reduced-rank adaptive algorithms according to the constant…
Computing the dominant eigenvalue is important in nuclear systems as it determines the stability of the system (i.e. whether the system is sub or supercritical). Recently, the work of Kusch, Whewell, McClarren and Frank \cite{KWMF} showed…
In this paper, we consider optimal low-rank regularized inverse matrix approximations and their applications to inverse problems. We give an explicit solution to a generalized rank-constrained regularized inverse approximation problem,…
Dynamical low-rank (DLR) approximation has gained interest in recent years as a viable solution to the curse of dimensionality in the numerical solution of kinetic equations including the Boltzmann and Vlasov equations. These methods…
In this paper, we extend the dynamical low-rank approximation method to the space of finite signed measures. Under this framework, we derive stochastic low-rank dynamics for stochastic differential equations (SDEs) coming from classical…
This paper introduces a novel variational Bayesian method that integrates Tucker decomposition for efficient high-dimensional inverse problem solving. The method reduces computational complexity by transforming variational inference from a…
The dynamical low-rank approximation of time-dependent matrices is a low-rank factorization updating technique. It leads to differential equations for factors of the matrices, which need to be solved numerically. We propose and analyze a…
Fine-tuning large pre-trained vision foundation models in a parameter-efficient manner is critical for downstream vision tasks, considering the practical constraints of computational and storage costs. Low-rank adaptation (LoRA) is a…
The latest advancements in neural image compression show great potential in surpassing the rate-distortion performance of conventional standard codecs. Nevertheless, there exists an indelible domain gap between the datasets utilized for…
In this paper, we study the problem of image recovery from given partial (corrupted) observations. Recovering an image using a low-rank model has been an active research area in data analysis and machine learning. But often, images are not…
Parameter-efficient tuning methods such as LoRA could achieve comparable performance to model tuning by tuning a small portion of the parameters. However, substantial computational resources are still required, as this process involves…
We present a unified theoretical framework for parametric low-rank approximation, a research area devoted to the development of efficient algorithms that act as adaptive alternatives of traditional methods such as Singular Value…
Continual learning in Neural Machine Translation (NMT) faces the dual challenges of catastrophic forgetting and the high computational cost of retraining. This study establishes Low-Rank Adaptation (LoRA) as a parameter-efficient framework…