Related papers: Second-order robust parallel integrators for dynam…

Numerous applications in data mining and machine learning require recovering a matrix of minimal rank. Robust principal component analysis (RPCA) is a general framework for handling this kind of problems. Nuclear norm based convex surrogate…

This work studies the linear approximation of high-dimensional dynamical systems using low-rank dynamic mode decomposition (DMD). Searching this approximation in a data-driven approach is formalised as attempting to solve a low-rank…

In this paper, we show that Low Rank Adaptation (LoRA) as originally introduced in Hu et al. (2021) leads to suboptimal finetuning of models with large width (embedding dimension). This is due to the fact that adapter matrices A and B in…

In this work, we introduce a quadratically convergent and dynamically consistent integrator specifically designed for the replicator dynamics. The proposed scheme combines a two-stage rational approximation with a normalization step to…

Low-rank approximation of a matrix by means of random sampling has been consistently efficient in its empirical studies by many scientists who applied it with various sparse and structured multipliers, but adequate formal support for this…

The dynamical low-rank (DLR) approximation is an efficient technique to approximate the solution to matrix differential equations. Recently, the DLR method was applied to radiation transport calculations to reduce memory requirements and…

We propose a numerical integrator for determining low-rank approximations to solutions of large-scale matrix differential equations. The considered differential equations are semilinear and stiff. Our method consists of first splitting the…

Neural networks have achieved tremendous success in a large variety of applications. However, their memory footprint and computational demand can render them impractical in application settings with limited hardware or energy resources. In…

We propose efficient parallel algorithms and implementations on shared memory architectures of LU factorization over a finite field. Compared to the corresponding numerical routines, we have identified three main difficulties specific to…

Low-rank adaptation (LoRA) has emerged as a widely adopted parameter-efficient fine-tuning method in deep transfer learning, due to its reduced number of trainable parameters and lower memory requirements enabled by Burer-Monteiro…

We perform an error analysis of a fully discretised Streamline Upwind Petrov Galerkin Dynamical Low Rank (SUPG-DLR) method for random time-dependent advection-dominated problems. The time integration scheme has a splitting-like nature,…

Robust principal component analysis is an important representative method in data analysis. It is usually viewed as an optimization problem involving the rank and $\ell_0$-norm of matrices. In this paper, we study the rank and $\ell_0$…

With the ever-growing size of pretrained models (PMs), fine-tuning them has become more expensive and resource-hungry. As a remedy, low-rank adapters (LoRA) keep the main pretrained weights of the model frozen and just introduce some…

The increasing complexity of deep learning recommendation models (DLRM) has led to a growing need for large-scale distributed systems that can efficiently train vast amounts of data. In DLRM, the sparse embedding table is a crucial…

Variational integrators are momentum-preserving and symplectic numerical methods used to propagate the evolution of Hamiltonian systems. In this paper, we introduce a new class of variational integrators that achieve fourth-order…

Low-rank modeling has a lot of important applications in machine learning, computer vision and social network analysis. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has…

We extend our previous work on a semi-Lagrangian adaptive rank (SLAR) integrator, in the finite difference framework for nonlinear Vlasov-Poisson systems, to the general high-order tensor setting. The proposed scheme retains the high-order…

Dominance-based Rough Set Approach (DRSA), as the extension of Pawlak's Rough Set theory, is effective and fundamentally important in Multiple Criteria Decision Analysis (MCDA). In previous DRSA models, the definitions of the upper and…

This paper addresses the problem of efficiently computing higher-order variational integrators in simulation and trajectory optimization of mechanical systems as those often found in robotic applications. We develop $O(n)$ algorithms to…

Balancing between computational efficiency and sample efficiency is an important goal in reinforcement learning. Temporal difference (TD) learning algorithms stochastically update the value function, with a linear time complexity in the…