Related papers: OCTOPUS: Optimized KV Cache for Transformers via O…

Post-training quantization (PTQ) has become a crucial tool for reducing the memory and compute costs of modern deep neural networks, including large language models (LLMs). Among PTQ algorithms, the OPTQ framework-also known as GPTQ-has…

The key-value (KV) cache in large language models presents a significant memory bottleneck during inference, growing linearly with sequence length and often exceeding the memory footprint of model weights themselves. We implement and…

Neural compression has brought tremendous progress in designing lossy compressors with good rate-distortion (RD) performance at low complexity. Thus far, neural compression design involves transforming the source to a latent vector, which…

The ever-growing size of neural networks poses serious challenges on resource-constrained devices, such as embedded sensors. Compression algorithms that reduce their size can mitigate these problems, provided that model performance stays…

Recent advancements in Audio-Video Large Language Models (AV-LLMs) have enhanced their capabilities in tasks like audio-visual question answering and multimodal dialog systems. Video and audio introduce an extended temporal dimension,…

Performing experiments on small-scale quantum computers is certainly a challenging endeavor. Many parameters need to be optimized to achieve high-fidelity operations. This can be done efficiently for operations acting on single qubits as…

Quantum error-correcting codes are constructed that embed a finite-dimensional code space in the infinite-dimensional Hilbert space of a system described by continuous quantum variables. These codes exploit the noncommutative geometry of…

For unsupervised data-dependent hashing, the two most important requirements are to preserve similarity in the low-dimensional feature space and to minimize the binary quantization loss. A well-established hashing approach is Iterative…

Large language models have revolutionized natural language processing but face significant challenges of high storage and runtime costs, due to the transformer architecture's reliance on self-attention, particularly the large KV cache for…

Large language models (LLMs) inference relies heavily on KV-caches to accelerate autoregressive decoding, but the resulting memory footprint grows rapidly with sequence length, posing significant efficiency challenges. Current KV-cache…

Model quantization has become a crucial technique to address the issues of large memory consumption and long inference times associated with LLMs. Mixed-precision quantization, which distinguishes between important and unimportant…

Data clipping is crucial in reducing noise in quantization operations and improving the achievable accuracy of quantization-aware training (QAT). Current practices rely on heuristics to set clipping threshold scalars and cannot be shown to…

Information reconciliation is crucial for continuous-variable quantum key distribution (CV-QKD) because its performance affects the secret key rate and maximal secure transmission distance. Fixed-rate error correction codes limit the…

In this paper, the quaternion matrix UTV (QUTV) decomposition and quaternion tensor UTV (QTUTV) decomposition are proposed. To begin, the terms QUTV and QTUTV are defined, followed by the algorithms. Subsequently, by employing random…

The high-fidelity storage of quantum information is crucial for quantum computation and communication. Many experimental platforms for these applications exhibit highly biased noise, with good resilience to spin depolarisation undermined by…

The impressive capabilities of Large Language Models (LLMs) come at the cost of substantial computational resources during deployment. While KV Cache can significantly reduce recomputation during inference, it also introduces additional…

Combining tunable metasurfaces with chaotic cavities opens new avenues for finely tailored dynamic control of microwaves with programmable coding metacavities (PCMs). There is currently a strong interest in utilizing PCMs to overcome the…

We work out a theory of approximate quantum error correction that allows us to derive a general lower bound for the entanglement fidelity of a quantum code. The lower bound is given in terms of Kraus operators of the quantum noise. This…

Quantizing the key-value (KV) cache is a promising strategy for improving the inference efficiency of large language models (LLMs). However, aggressive quantization to very low precision (e.g., 2 bits) introduces significant errors in the…

Key-value (KV) caching has emerged as a crucial optimization technique for accelerating inference in large language models (LLMs). By allowing the attention operation to scale linearly rather than quadratically with the total sequence…