Related papers: A Tighter Complexity Analysis of SparseGPT
Small, highly trained, open-source large language models are widely used due to their inference efficiency, but further improving their quality remains a challenge. Sparse upcycling is a promising approach that transforms a pretrained dense…
Integer programs with m constraints are solvable in pseudo-polynomial time in $\Delta$, the largest coefficient in a constraint, when m is a fixed constant. We give a new algorithm with a running time of $O(\sqrt{m}\Delta)^{2m} + O(nm)$,…
Orthogonal matching pursuit (OMP) is a greedy algorithm widely used for the recovery of sparse signals from compressed measurements. In this paper, we analyze the number of iterations required for the OMP algorithm to perform exact recovery…
Pruhs and Woeginger prove the existence of FPTAS's for a general class of minimization and maximization subset selection problems. Without losing generality from the original framework, we prove how better asymptotic worst-case running…
Sparse signal recovery deals with finding the sparsest solution of an under-determined linear system $\vx = \mQ\vs$. In this paper, we propose a novel greedy approach to addressing the challenges from such a problem. Such an approach is…
In this work, we use Deep Gaussian Processes (DGPs) as statistical surrogates for stochastic processes with complex distributions. Conventional inferential methods for DGP models can suffer from high computational complexity as they require…
A wide range of problems in computational science and engineering require estimation of sparse eigenvectors for high dimensional systems. Here, we propose two variants of the Truncated Orthogonal Iteration to compute multiple leading…
Reducing the computational cost of running large scale neural networks using sparsity has attracted great attention in the deep learning community. While much success has been achieved in reducing FLOP and parameter counts while maintaining…
Orthogonal Matching Pursuit (OMP) has been a powerful method in sparse signal recovery and approximation. However, OMP suffers computational issues when the signal has a large number of non-zeros. This paper advances OMP and its extension…
The simplex method for linear programming is known to be highly efficient in practice, and understanding its performance from a theoretical perspective is an active research topic. The framework of smoothed analysis, first introduced by…
In this paper we provide new randomized algorithms with improved runtimes for solving linear programs with two-sided constraints. In the special case of the minimum cost flow problem on $n$-vertex $m$-edge graphs with integer…
Adaptive learning is necessary for non-stationary environments where the learning machine needs to forget past data distribution. Efficient algorithms require a compact model update to not grow in computational burden with the incoming data…
Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace existing Deep Neural Networks (DNNs). Despite the remarkable…
We consider the problem of multiplying sparse matrices (over a semiring) where the number of non-zero entries is larger than main memory. In the classical paper of Hong and Kung (STOC '81) it was shown that to compute a product of dense $U…
Clique tree conversion solves large-scale semidefinite programs by splitting an $n\times n$ matrix variable into up to $n$ smaller matrix variables, each representing a principal submatrix of up to $\omega\times\omega$. Its fundamental…
Given $n$ jobs with processing times $p_1,\dotsc,p_n\in\mathbb N$ and $m\le n$ machines with speeds $s_1,\dotsc,s_m\in\mathbb N$ our goal is to allocate the jobs to machines minimizing the makespan. We present an algorithm that solves the…
Sparse tensor programs are essential in deep learning and graph analytics, driving the need for optimized processing. To meet this demand, specialized hardware accelerators are being developed. Optimizing these programs for accelerators is…
In the restricted shortest paths problem, we are given a graph $G$ whose edges are assigned two non-negative weights: lengths and delays, a source $s$, and a delay threshold $D$. The goal is to find, for each target $t$, the length of the…
The {\it matrix-chain multiplication} problem is a classic problem that is widely taught to illustrate dynamic programming. The textbook solution runs in $\theta(n^3)$ time. However, there is a complex $O(n \log n)$-time method \cite{HU82},…
Sparse matrix multiplication is an important kernel for large-scale graph processing and other data-intensive applications. In this paper, we implement various asynchronous, RDMA-based sparse times dense (SpMM) and sparse times sparse…