Related papers: Very Low-Rate Variable-Length Channel Quantization…

The impressive capabilities of Large Language Models (LLMs) come at the cost of substantial computational resources during deployment. While KV Cache can significantly reduce recomputation during inference, it also introduces additional…

Data-free quantization (DFQ) is a technique that creates a lightweight network from its full-precision counterpart without the original training data, often through a synthetic dataset. Although several DFQ methods have been proposed for…

Classically, communication systems are designed assuming perfect channel state information at the receiver and/or transmitter. However, in many practical situations, only an estimate of the channel is available that differs from the true…

Traditional post-training quantization (PTQ) is considered an effective approach to reduce model size and accelerate inference of large-scale language models (LLMs). However, existing low-rank PTQ methods require costly fine-tuning to…

In a networked control system, quantization is inevitable to transmit control and measurement signals. While uniform quantizers are often used in practical systems, the overloading, which is due to the limitation on the number of bits in…

Variational Quantum Algorithms (VQA) are promising to demonstrate quantum advantages on near-term devices. Designing ansatz, a variational circuit with parameterized gates, is of paramount importance for VQA as it lays the foundation for…

This paper deals with the challenging problem of spectrum sensing in cognitive radio. We consider a stochastic system model where the the Primary User (PU) transmits a periodic signal over fading channels. The effect of frequency offsets…

The outage probability limit is a fundamental and achievable lower bound on the word error rate of coded communication systems affected by fading. This limit is mainly determined by two parameters: the diversity order and the coding gain.…

Quantization is a fundamental optimization for many machine-learning use cases, including compressing gradients, model weights and activations, and datasets. The most accurate form of quantization is \emph{adaptive}, where the error is…

We prove achievability of the recently characterized quadratic Gaussian rate-distortion function (RDF) subject to the constraint that the distortion is uncorrelated to the source. This result is based on shaped dithered lattice quantization…

We study the performance of polarizing codes over a degraded symmetric wiretap channel under a total variation distance (TVD) secrecy constraint. We show that the leakage can be bounded by the sum of the TVDs of the bit-channels…

Cell-free Massive multiple-input multiple-output (MIMO) is considered, where distributed access points (APs) multiply the received signal by the conjugate of the estimated channel, and send back a quantized version of this weighted signal…

This paper addresses channel estimation and data equalization on frequency-selective 1-bit quantized Multiple Input-Multiple Output (MIMO) systems. No joint processing or Channel State Information is assumed at the transmitter, and…

The problem of dirty paper coding (DPC) over the (multi-antenna) fading dirty paper channel (FDPC) Y = H(X + S) + Z is considered when there is imperfect knowledge of the channel state information H at the transmitter (CSIT). The case of…

Visual tokenizers are fundamental to image generation. They convert visual data into discrete tokens, enabling transformer-based models to excel at image generation. Despite their success, VQ-based tokenizers like VQGAN face significant…

Vision-Language Models (VLMs) such as CLIP have revolutionized zero-shot classification and safety-critical tasks, including Out-of-Distribution (OOD) detection. However, their high computational cost hinders efficient real-world…

Vector quantization is common in deep models, yet its hard assignments block gradients and hinder end-to-end training. We propose DiVeQ, which treats quantization as adding an error vector that mimics the quantization distortion, keeping…

Post-training quantization (PTQ) of large language models (LLMs) holds the promise in reducing the prohibitive computational cost at inference time. Quantization of all weight, activation and key-value (KV) cache tensors to 4-bit without…

This paper considers delay-limited communication over quasi-static fading channels under a long-term power constraint. A sequence of length-$n$ delay-limited codes for a quasi-static fading channel is said to be capacity-achieving if the…

Long Chain-of-Thought (CoT) reasoning has significantly advanced the capabilities of Large Language Models (LLMs), but this progress is accompanied by substantial memory and latency overhead from the extensive Key-Value (KV) cache. Although…