Related papers: Parallel QR Factorization of Block Low-Rank Matric…
Low rank approximation of matrices has been well studied in literature. Singular value decomposition, QR decomposition with column pivoting, rank revealing QR factorization (RRQR), Interpolative decomposition etc are classical deterministic…
Enumerating simple cycles has important applications in computational biology, network science, and financial crime analysis. In this work, we focus on parallelising the state-of-the-art simple cycle enumeration algorithms by Johnson and…
Many matrices associated with fast transforms posess a certain low-rank property characterized by the existence of several block partitionings of the matrix, where each block is of low rank. Provided that these partitionings are known,…
Dense linear layers are the dominant computational bottleneck in foundation models. Identifying more efficient alternatives to dense matrices has enormous potential for building more compute-efficient models, as exemplified by the success…
This paper investigates reduced complexity neural network (NN) based architectures for equalization over the two-dimension magnetic recording (TDMR) digital communication channel for data storage. We use realistic waveforms measured from a…
Kernel logistic regression (KLR) is a widely used supervised learning method for binary and multi-class classification, which provides estimates of the conditional probabilities of class membership for the data points. Unlike other kernel…
We present an efficient algorithm for the least squares parameter fitting optimized for component separation in multi-frequency CMB experiments. We sidestep some of the problems associated with non-linear optimization by taking advantage of…
Despite their tremendous success and versatility, Deep Neural Networks (DNNs) such as Large Language Models (LLMs) suffer from inference inefficiency and rely on advanced computational infrastructure. To address these challenges and make…
Some fast algorithms for computing the eigenvalues of a block companion matrix $A = U + XY^H$, where $U\in \mathbb C^{n\times n}$ is unitary block circulant and $X, Y \in\mathbb{C}^{n \times k}$, have recently appeared in the literature.…
The high inference demands of transformer-based Large Language Models (LLMs) pose substantial challenges in their deployment. To this end, we introduce Neural Block Linearization (NBL), a novel framework for accelerating transformer model…
The increasing size of transformer-based models in NLP makes the question of compressing them important. In this work, we present a comprehensive analysis of factorization based model compression techniques. Specifically, we focus on…
This study introduces a new normalization layer termed Batch Layer Normalization (BLN) to reduce the problem of internal covariate shift in deep neural network layers. As a combined version of batch and layer normalization, BLN adaptively…
A square-root-free matrix QR decomposition (QRD) scheme was rederived in [1] based on [2] to simplify computations when solving least-squares (LS) problems on embedded systems. The scheme of [1] aims at eliminating both the square-root and…
We propose a clustering-based generalized low rank approximation method, which takes advantage of appealing features from both the generalized low rank approximation of matrices (GLRAM) and cluster analysis. It exploits a more general form…
The classic rank-revealing QR factorization factorizes a matrix $A$ as $AP=QR$ where $P$ permutes the columns of $A$, $Q$ is an orthogonal matrix, and $R$ is upper triangular with non-increasing diagonal entries. This is called…
The approximate minimum degree algorithm is widely used before numerical factorization to reduce fill-in for sparse matrices. While considerable attention has been given to the numerical factorization process, less focus has been placed on…
Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factors $W$ and $H$, for the given input matrix $A$, such that $A \approx W H$. NMF is a useful tool for many applications in different domains…
Multiple kernel methods less consider the intrinsic manifold structure of multiple kernel data and estimate the consensus kernel matrix with quadratic number of variables, which makes it vulnerable to the noise and outliers within multiple…
Low-Rank Representation (LRR) is arguably one of the most powerful paradigms for Multi-view spectral clustering, which elegantly encodes the multi-view local graph/manifold structures into an intrinsic low-rank self-expressive data…
Random Feature Methods (RFMs) and their variants such as extreme learning machine finite-basis physics-informed neural networks (ELM-FBPINNs) offer a scalable approach for solving partial differential equations (PDEs) by using localized,…