Related papers: When Quantization Is Free: An int4 KV Cache That O…

Reasoning models excel at complex tasks such as coding and mathematics, yet their inference is often slow and token-inefficient. To improve the inference efficiency, post-training quantization (PTQ) usually comes with the cost of large…

Growing context lengths in transformer-based language models have made the key-value (KV) cache a critical inference bottleneck. While many KV cache pruning methods have been proposed, they have not yet been adopted in major inference…

We introduce QUICK, a group of novel optimized CUDA kernels for the efficient inference of quantized Large Language Models (LLMs). QUICK addresses the shared memory bank-conflict problem of state-of-the-art mixed precision matrix…

We report on the quantum storage and retrieval of photonic polarization quantum bits onto and out of a solid state storage device. The qubits are implemented with weak coherent states at the single photon level, and are stored for 500 ns in…

Bulk diamond phonons have been shown to be a versatile platform for the generation, storage, and manipulation of high-bandwidth quantum states of light. Here we demonstrate a diamond quantum memory that stores, and releases on demand, an…

Recently, convolutional neural networks (CNN) have demonstrated impressive performance in various computer vision tasks. However, high performance hardware is typically indispensable for the application of CNN models due to the high…

Post-training quantization (PTQ) assumes that a well-converged model is a quantization-ready model. We show this assumption fails in a structured, measurable, and previously uncharacterized way. Using a calibration-free per-group INT4 probe…

Diffusion-based large language models (dLLMs) rely on bidirectional attention, which prevents lossless KV caching and requires a full forward pass at every denoising step. Existing approximate KV caching methods reduce this cost by…

Quantum computing has the potential to significantly speed up complex computational tasks, and arguably the most promising application area for near-term quantum computers is the simulation of quantum mechanics. To make the most of our…

In superconducting quantum information, machined aluminum superconducting cavities have proven to be a well-controlled, low-dissipation electromagnetic environment for quantum circuits such as qubits. They can possess large internal quality…

We propose Rotate, Clip, and Partition (RCP), a quantization-aware training (QAT) approach that first realizes extreme compression of LLMs with W2A4KV4(2-bit weight, 4-bit activation, and 4-bit KV cache) configuration. RCP integrates recent…

We introduce LogQuant, a groundbreaking 2-bit quantization technique for KV Cache in large language model (LLM) inference, delivering substantial memory savings while preserving superior performance. Previous methods either assume that…

Large Language Models (LLMs) demonstrate exceptional performance but entail significant memory and computational costs, restricting their practical deployment. While existing INT4/INT8 quantization reduces these costs, they often degrade…

We report the first end-to-end hardware-validated demonstration of a reversible Quantum Memory Matrix QMM imprint retrieval cycle. Using IBM Quantum back ends, we realize five imprint retrieval experiments that scale from a minimal…

The key-value (KV) cache in transformer models is a critical component for efficient decoding or inference, yet its memory demands scale poorly with sequence length, posing a major challenge for scalable deployment of large language models.…

Quantum machine learning is a rapidly growing field at the intersection of quantum computing and machine learning. In this work, we examine our quantum machine learning models, which are based on quantum support vector classification (QSVC)…

Recent advances in self-supervised learning and the Transformer architecture have significantly improved natural language processing (NLP), achieving remarkably low perplexity. However, the growing size of NLP models introduces a memory…

Catastrophic forgetting poses a fundamental challenge in continual learning, particularly when models are quantized for deployment efficiency. We systematically investigate the interplay between quantization precision (FP16, INT8, INT4) and…

Key-Value (KV) cache quantization has become a widely adopted optimization technique for efficient large language models (LLMs) inference by reducing KV cache memory usage and mitigating memory-bound constraints. Recent studies have…

Specialized edge accelerators rely on low-bit quantization, but vendor compilers differ in scaling, clipping, and kernel support, often as black boxes. The same floating-point (FP) checkpoint can therefore yield inconsistent accuracy across…