Related papers: Using Mixed Precision in Low-Synchronization Reort…
The cyclic block coordinate descent-type (CBCD-type) methods, which performs iterative updates for a few coordinates (a block) simultaneously throughout the procedure, have shown remarkable computational performance for solving strongly…
We consider the problem of recovering off-the-grid spikes from linear measurements. The state of the art Over-Parametrized Continuous Orthogonal Matching Pursuit (OP-COMP) with Projected Gradient Descent (PGD) successfully recovers those…
Large-scale L1-regularized loss minimization problems arise in high-dimensional applications such as compressed sensing and high-dimensional supervised learning, including classification and regression problems. High-performance algorithms…
Our goal in this paper is to clarify the relationship between the block Lanczos and the block conjugate gradient (BCG) algorithms. Under the full rank assumption for the block vectors, we show the one-to-one correspondence between the…
Low rank matrix approximations appear in a number of scientific computing applications. We consider the Nystr\"{o}m method for approximating a positive semidefinite matrix $A$. In the case that $A$ is very large or its entries can only be…
Gradient orthogonalization is a simple strategy that shows great utility in speeding up gradient descent. The Muon optimizer (Jordan, Jin, et al., 2024) combines gradient orthogonalization with first-order momentum and achieves significant…
The iteration complexity of the block-coordinate descent (BCD) type algorithm has been under extensive investigation. It was recently shown that for convex problems the classical cyclic BCGD (block coordinate gradient descent) achieves an…
We consider the approximate recovery of multivariate periodic functions from a discrete set of function values taken on a rank-$s$ integration lattice. The main result is the fact that any (non-)linear reconstruction algorithm taking…
Orthogonal Matching Pursuit (OMP) has been a powerful method in sparse signal recovery and approximation. However, OMP suffers computational issues when the signal has a large number of non-zeros. This paper advances OMP and its extension…
We present a mixed-precision benchmark called HPL-MxP that uses both a lower-precision LU factorization with a non-stationary iterative refinement based on GMRES. We evaluate the numerical stability of one of the methods of generating the…
Block Floating Point (BFP) arithmetic is currently seeing a resurgence in interest because it requires less power, less chip area, and is less complicated to implement in hardware than standard floating point arithmetic. This paper explores…
For greedy block sparse recovery where the sparsity level is unknown, we derive a stopping condition to stop the iteration process. Focused on the block orthogonal matching pursuit (BOMP) algorithm, we model the energy of residual signals…
Support for arithmetic in multiple precisions and number formats is becoming increasingly common in emerging high-performance architectures. From a computational scientist's perspective, our goal is to determine how and where we can safely…
Low-precision arithmetic has had a transformative effect on the training of neural networks, reducing computation, memory and energy requirements. However, despite its promise, low-precision arithmetic has received little attention for…
The joint bidiagonalization process of a matrix pair $\{A,L\}$ can be used to develop iterative regularization algorithms for large scale ill-posed problems in general-form Tikhonov regularization…
Mixed-precision algorithms have been proposed as a way for scientific computing to benefit from some of the gains seen for artificial intelligence (AI) on recent high performance computing (HPC) platforms. A few applications dominated by…
We present an entirely new geometric and probabilistic approach to synchronization of correspondences across multiple sets of objects or images. In particular, we present two algorithms: (1) Birkhoff-Riemannian L-BFGS for optimizing the…
Optimizing over the set of orthogonal matrices is a central component in problems like sparse-PCA or tensor decomposition. Unfortunately, such optimization is hard since simple operations on orthogonal matrices easily break orthogonality,…
In this paper, we propose two novel modulation concepts based on a simple maximum distance separable (MDS) code { and show that these concepts can achieve better error performance than index modulation (IM) and related schemes.} In the…
We study the statistical estimation problem of orthogonal group synchronization and rotation group synchronization. The model is $Y_{ij} = Z_i^* Z_j^{*T} + \sigma W_{ij}\in\mathbb{R}^{d\times d}$ where $W_{ij}$ is a Gaussian random matrix…