Related papers: Independent Low-Rank Matrix Analysis Based on Para…
Fine-tuning has proven to be highly effective in adapting pre-trained models to perform better on new desired tasks with minimal data samples. Among the most widely used approaches are reparameterization methods, which update a target…
Recommendation systems often rely on point-wise loss metrics such as the mean squared error. However, in real recommendation settings only few items are presented to a user. This observation has recently encouraged the use of rank-based…
We develop computational methods for approximating the solution of a linear multi-term matrix equation in low rank. We follow an alternating minimization framework, where the solution is represented as a product of two matrices, and…
Sparse PCA is a widely used technique for high-dimensional data analysis. In this paper, we propose a new method called low-rank principal eigenmatrix analysis. Different from sparse PCA, the dominant eigenvectors are allowed to be dense…
We study low-rank matrix estimation for a generic inhomogeneous output channel through which the matrix is observed. This generalizes the commonly considered spiked matrix model with homogeneous noise to include for instance the dense…
This paper presents a randomized algorithm for computing the near-optimal low-rank dynamic mode decomposition (DMD). Randomized algorithms are emerging techniques to compute low-rank matrix approximations at a fraction of the cost of…
We describe an iterative method to optimize the multi-scale entanglement renormalization ansatz (MERA) for the low-energy subspace of local Hamiltonians on a D-dimensional lattice. For translation invariant systems the cost of this…
Effectively integrating molecular graph structures with Large Language Models (LLMs) is a key challenge in drug discovery. Most existing multi-modal alignment methods typically process these structures by fine-tuning the LLM or adding a…
Bilinear matrix inequality (BMI) problems in system and control designs are investigated in this paper. A solution method of reduction of variables (MRVs) is proposed. This method consists of a principle of variable classification, a…
Over the last years, novel low-complexity approaches to the equalization of MIMO channels have gained much attention. Thereby, methods based on lattice basis reduction are of special interest, as they achieve the optimum diversity order. In…
In addressing the computational and memory demands of fine-tuning Large Language Models(LLMs), we propose LoRA-SP(Streamlined Partial Parameter Adaptation), a novel approach utilizing randomized half-selective parameter freezing within the…
Large language models are often adapted using parameter-efficient techniques such as Low-Rank Adaptation (LoRA), formulated as $y = W_0x + BAx$, where $W_0$ is the pre-trained parameters and $x$ is the input to the adapted layer. While…
Deploying transformer-based neural networks on resource-constrained edge devices presents a significant challenge. This challenge is often addressed through various techniques, such as low-rank approximation and mixed-precision…
In this work and its accompanying Part II [1], we develop an accelerated algorithmic framework, DAMA (Decentralized Accelerated Minimax Approach), for nonconvex Polyak-Lojasiewicz minimax optimization over decentralized multi-agent…
Ranking models, i.e., coarse-ranking and fine-ranking models, serve as core components in large-scale recommendation systems, responsible for scoring massive item candidates based on user preferences. To meet the stringent latency…
Computing high-quality graph partitions is a challenging problem with numerous applications. In this paper, we present a novel meta-heuristic for the balanced graph partitioning problem. Our approach is based on integer linear programs that…
This paper is concerned with the Taylor-reduced basis method (Taylor-RBM) for the efficient approximation of eigenspaces of large scale parametric Hermitian matrices. The Taylor-RBM is a local model order reduction method, which constructs…
In this paper, we show how a resource allocation problem can be solved through Integer Linear Programming (ILP). A detailed illustrative example is presented, together with an exhaustive overview of the mathematical model. The size of the…
This paper studies the problem of recovering a low-rank matrix from several noisy random linear measurements. We consider the setting where the rank of the ground-truth matrix is unknown a priori and use an objective function built from a…
The computation of the dominant eigenpair for symmetric positive semidefinite matrices is fundamental in numerical optimization. This work shifts the paradigm from the classical Rayleigh quotient to an unconstrained difference formulation,…