Related papers: Robust Implicit Adaptive Low Rank Time-Stepping Me…

A numerical dynamical low-rank approximation (DLRA) scheme for the solution of the Vlasov-Poisson equation is presented. Based on the formulation of the DLRA equations as Friedrichs' systems in a continuous setting, it combines recently…

Radiation transport problems are posed in a high-dimensional phase space, limiting the use of finely resolved numerical simulations. An emerging tool to efficiently reduce computational costs and memory footprint in such settings is…

A matrix algorithm runs superfast (aka at sublinear cost) if it involves much fewer flops and memory cells than an input matrix has entries. Big Data are frequently represented by matrices of immense sizes that cannot be handled directly…

This paper introduces a novel computational approach termed the Reduced Augmentation Implicit Low-rank (RAIL) method by investigating two predominant research directions in low-rank solutions to time-dependent partial differential equations…

We propose a high order adaptive-rank implicit integrators for stiff time-dependent PDEs, leveraging extended Krylov subspaces to efficiently and adaptively populate low-rank solution bases. This allows for the accurate representation of…

In algorithms for solving optimization problems constrained to a smooth manifold, retractions are a well-established tool to ensure that the iterates stay on the manifold. More recently, it has been demonstrated that retractions are a…

Explicit step-truncation tensor methods have recently proven successful in integrating initial value problems for high-dimensional partial differential equations (PDEs). However, the combination of non-linearity and stiffness may introduce…

In this work (Part I), we study three time-discretization procedures of the Dynamical Low-Rank Approximation (DLRA) of high-dimensional stochastic differential equations (SDEs). Specifically, we consider the Dynamically Orthogonal (DO)…

Dynamical low-rank approximation (DLRA) is an emerging tool for reducing computational costs and provides memory savings when solving high-dimensional problems. In this work, we propose and analyze a semi-implicit dynamical low-rank…

We consider dynamical low-rank approximation (DLRA) for the numerical simulation of Vlasov--Poisson equations based on separation of space and velocity variables, as proposed in several recent works. The standard approach for the time…

We perform an error analysis of a fully discretised Streamline Upwind Petrov Galerkin Dynamical Low Rank (SUPG-DLR) method for random time-dependent advection-dominated problems. The time integration scheme has a splitting-like nature,…

Dynamic Rank Reinforcement Learning (DR-RL) approximations rely on static rank assumptions, limiting their flexibility across diverse linguistic contexts. Our method dynamically modulates ranks based on real-time sequence dynamics,…

The dynamical low-rank approximation of time-dependent matrices is a low-rank factorization updating technique. It leads to differential equations for factors of the matrices, which need to be solved numerically. We propose and analyze a…

Low-Rank Adaptation (LoRA), which leverages the insight that model updates typically reside in a low-dimensional space, has significantly improved the training efficiency of Large Language Models (LLMs) by updating neural network layers…

We propose and analyse a numerical integrator that computes a low-rank approximation to large time-dependent matrices that are either given explicitly via their increments or are the unknown solution to a matrix differential equation.…

Low-Rank Adaptation (LoRA) is a crucial method for efficiently fine-tuning large language models (LLMs), with its effectiveness influenced by two key factors: rank selection and weight initialization. While numerous LoRA variants have been…

This paper proposes a new framework for computing low-rank solutions to nonlinear matrix equations arising from spatial discretization of nonlinear partial differential equations: low-rank Anderson acceleration (lrAA). lrAA is an adaptation…

Low-rank methods have emerged as a promising strategy for reducing the memory footprint and computational cost of discrete-ordinates discretizations of the radiative transfer equation (RTE). However, most existing rank-adaptive approaches…

In the fields of control theory and machine learning, the dynamic low-rank approximation for large-scale matrices has received substantial attention. Considering large-scale semilinear stiff matrix differential equations, we propose…

Low-rank adaptation is effective partly because downstream updates lie in a low-dimensional subspace, but the latent rank coordinates of LoRA are not identifiable: any invertible reparameterization of the adapter factors leaves the weight…