Related papers: Maximizing the Bang Per Bit
Homomorphic Encryption (HE) enables users to securely outsource both the storage and computation of sensitive data to untrusted servers. Not only does HE offer an attractive solution for security in cloud systems, but lattice-based HE…
Stateful optimizers maintain gradient statistics over time, e.g., the exponentially smoothed sum (SGD with momentum) or squared sum (Adam) of past gradient values. This state can be used to accelerate optimization compared to plain…
Parallel implementations of stochastic gradient descent (SGD) have received significant research attention, thanks to excellent scalability properties of this algorithm, and to its efficiency in the context of training deep neural networks.…
Quantum field theories underlie all of our understanding of the fundamental forces of nature. The are relatively few first principles approaches to the study of quantum field theories [such as quantum chromodynamics (QCD) relevant to the…
Attention-based large language models (LLMs) have transformed modern AI applications, but the quadratic cost of self-attention imposes significant compute and memory overhead. Dynamic sparsity (DS) attention mitigates this, yet its hardware…
Benders decomposition is widely used to solve large mixed-integer problems. This paper takes advantage of machine learning and proposes enhanced variants of Benders decomposition for solving two-stage stochastic security-constrained unit…
Multimodal Large Language Models (MLLMs) have advanced unified reasoning over text, images, and videos, but their inference is hindered by the rapid growth of key-value (KV) caches. Each visual input expands into thousands of tokens,…
Here we present the cuLGT code for gauge fixing in lattice gauge field theories with graphic processing units (GPUs). Implementations for SU(3) Coulomb, Landau and maximally Abelian gauge fixing are available and the overrelaxation,…
The most scalable proposed methods of simulating lattice fermions on noisy quantum computers employ encodings that eliminate nonlocal operators using a constant factor more qubits and a nontrivial stabilizer group. In this work, we…
We present thread-safe, highly-optimized lattice Boltzmann implementations, specifically aimed at exploiting the high memory bandwidth of GPU-based architectures. At variance with standard approaches to LB coding, the proposed strategy,…
The design complexity of CNNs has been steadily increasing to improve accuracy. To cope with the massive amount of computation needed for such complex CNNs, the latest solutions utilize blocking of an image over the available dimensions and…
Large-scale distributed optimization is of great importance in various applications. For data-parallel based distributed learning, the inter-node gradient communication often becomes the performance bottleneck. In this paper, we propose the…
As deep neural networks (DNNs) see increased deployment on mobile and edge devices, optimizing model efficiency has become crucial. Mixed-precision quantization is widely favored, as it offers a superior balance between efficiency and…
High-rate quantum LDPC (qLDPC) codes reduce memory overhead by densely packing many logical qubits into a single block of physical qubits. Here we extend this concept to high-rate computation by constructing \emph{batched} fault-tolerant…
LLM decoding is bottlenecked for large batches and long contexts by loading the key-value (KV) cache from high-bandwidth memory, which inflates per-token latency, while the sequential nature of decoding limits parallelism. We analyze the…
It is well-known that molecular dynamics integrators, which are used for lattice quantum chromodynamics (QCD), suffer from instabilities and possess a rather low order of the accuracy. Hence, it is highly desirable to construct a new class…
Rapid growth in scientific data and a widening gap between computational speed and I/O bandwidth make it increasingly infeasible to store and share all data produced by scientific simulations. Instead, we need methods for reducing data…
Mixed-precision algorithms have been proposed as a way for scientific computing to benefit from some of the gains seen for artificial intelligence (AI) on recent high performance computing (HPC) platforms. A few applications dominated by…
The quantum stabilizer formalism became foundational for understanding error correction soon after the realization of the first useful quantum error correction codes. Stabilizers provide a way to describe sets of quantum states which are…
We accelerated an ab-initio molecular QMC calculation by using GPGPU. Only the bottle-neck part of the calculation is replaced by CUDA subroutine and performed on GPU. The performance on a (single core CPU + GPU) is compared with that on a…