Related papers: Dynamical Low-Rank Approximation for Stochastic Di…

With the ever-growing size of pretrained models (PMs), fine-tuning them has become more expensive and resource-hungry. As a remedy, low-rank adapters (LoRA) keep the main pretrained weights of the model frozen and just introduce some…

Parareal is a well-studied algorithm for numerically integrating systems of time-dependent differential equations by parallelising the temporal domain. Given approximate initial values at each temporal sub-interval, the algorithm locates a…

Conventional Low-Rank Adaptation (LoRA) methods employ a fixed rank, imposing uniform adaptation across transformer layers and attention heads despite their heterogeneous learning dynamics. This paper introduces Adaptive Rank Dynamic LoRA…

It is generally recognized that finite learning rate (LR), in contrast to infinitesimal LR, is important for good generalization in real-life deep nets. Most attempted explanations propose approximating finite-LR SGD with Ito Stochastic…

We introduce two novel interpolatory dynamical low-rank (DLR) approximation methods for the efficient time integration of the Boltzmann-BGK equation. Both methods overcome limitations of classic DLR schemes based on orthogonal projections…

In this paper we present the theoretical framework needed to justify the use of a kernel-based collocation method (meshfree approximation method) to estimate the solution of high-dimensional stochastic partial differential equations…

In this paper, we develop a low-rank method with high-order temporal accuracy using spectral deferred correction (SDC) to compute linear matrix differential equations. In [1], a low rank numerical method is proposed to correct the modeling…

Dynamical low-rank algorithms are a class of numerical methods that compute low-rank approximations of dynamical systems. This is accomplished by projecting the dynamics onto a low-dimensional manifold and writing the solution directly in…

Stochastic differential equations (SDEs) and the Kolmogorov partial differential equations (PDEs) associated to them have been widely used in models from engineering, finance, and the natural sciences. In particular, SDEs and Kolmogorov…

Low-rank matrix approximation (LRMA) is a powerful technique for signal processing and pattern analysis. However, its potential for data compression has not yet been fully investigated in the literature. In this paper, we propose sparse…

Dominance-based Rough Set Approach (DRSA), as the extension of Pawlak's Rough Set theory, is effective and fundamentally important in Multiple Criteria Decision Analysis (MCDA). In previous DRSA models, the definitions of the upper and…

We propose and analyze a second-order Strang splitting method for a class of stiff matrix differential equations with Sylvester-type structure. The method splits the dynamics into a stiff linear part, treated exactly via matrix…

This paper presents a randomized algorithm for computing the near-optimal low-rank dynamic mode decomposition (DMD). Randomized algorithms are emerging techniques to compute low-rank matrix approximations at a fraction of the cost of…

The dynamics of rough differential equations (RDEs) has recently received a lot of interest. For example, the existence of local random center manifolds for RDEs has been established. In this work, we present an approximation for local…

Conventional direction of arrival (DOA) estimation algorithms suffer from performance degradation due to antenna pattern distortion and substantial computational complexity in real-time execution. The support vector regression (SVR)…

Asynchronous stochastic gradient descent (ASGD) is a popular parallel optimization algorithm in machine learning. Most theoretical analysis on ASGD take a discrete view and prove upper bounds for their convergence rates. However, the…

In this paper, we consider low-rank approximations for the solutions to the stochastic Helmholtz equation with random coefficients. A Stochastic Galerkin finite element method is used for the discretization of the Helmholtz problem.…

Stochastic differential equations (SDEs) offer powerful and accessible mathematical models for capturing both deterministic and probabilistic aspects of dynamic behavior across a wide range of physical, financial, and social systems.…

In this paper, we consider multi-stage stochastic optimization problems with convex objectives and conic constraints at each stage. We present a new stochastic first-order method, namely the dynamic stochastic approximation (DSA) algorithm,…

Dynamic optimization problems involving discrete decisions have several applications, yet lead to challenging optimization problems that must be addressed efficiently. Combining discrete variables with potentially nonlinear constraints…