Related papers: PolarQuant: Leveraging Polar Transformation for Ef…

The polarization decomposition of arbitrary binary-input memoryless channels (BMCs) is studied in this work. By introducing the polarization factor (PF), defined in terms of the conditional entropy of the channel output under various input…

Arikan's recursive code construction is designed to polarize a collection of memoryless channels into a set of good and a set of bad channels, and it can be efficiently decoded using successive cancellation. It was recently shown that the…

Quantum memory for flying optical qubits is a key enabler for a wide range of applications in quantum information science and technology. A critical figure of merit is the overall storage-and-retrieval efficiency. So far, despite the recent…

Quantization can be used to form new vectors/matrices with shared values close to the original. In recent years, the popularity of scalar quantization for value-sharing applications has been soaring as it has been found huge utilities in…

In 2008 Arikan proposed polar coding [arXiv:0807.3917] which we summarize as follows: (a) From the root channel $W$ synthesize recursively a series of channels $W_N^{(1)},\dotsc,W_N^{(N)}$. (b) Select sophisticatedly a subset $A$ of…

Large language models (LLMs) have shown strong performance across diverse tasks, but their inference with long input contexts is bottlenecked by memory size and bandwidth. The Key-Value (KV) cache size grows linearly with sequence length…

Large Language Models (LLMs) suffer inference-time memory bottlenecks dominated by the attention Key-Value (KV) cache, which scales with model size and context length. While KV-cache quantization alleviates this cost, bit allocation between…

Weight quantisation is an essential technique for enabling efficient training and deployment of modern deep learning models. However, the recipe book of quantisation formats is large and formats are often chosen empirically. In this paper,…

Polar encoding, described by Arikan in IEEE Transactions on Information Theory, Vol. 55, No. 7, July 2009, was a milestone for telecommunications. A Polar code distributes information among high and low-capacity channels, showing the…

Polar coding is a recently proposed coding technique that can provably achieve the channel capacity. The polar code structure, which is based on the original 2x2 generator matrix, polarises the channels, i.e., a portion of the channel…

This paper describes efficient algorithms for computing rank-revealing factorizations of matrices that are too large to fit in RAM, and must instead be stored on slow external memory devices such as solid-state or spinning disk hard drives…

Video large language models (VideoLLMs) have demonstrated the capability to process longer video inputs and enable complex reasoning and analysis. However, due to the thousands of visual tokens from the video frames, the key-value (KV)…

Quantum Computing in the Noisy Intermediate-Scale Quantum (NISQ) era has shown promising applications in machine learning, optimization, and cryptography. Despite the progress, challenges persist due to system noise, errors, and decoherence…

Network quantization has emerged as one of the most practical model compression techniques, which significantly reduces a model's memory and compute consumption by mapping floating-point numbers to low-bit representations. However, existing…

Quantization of foundational models (FMs) is significantly more challenging than traditional DNNs due to the emergence of large magnitude values called outliers. Existing outlier-aware algorithm-architecture co-design techniques either use…

Applications of massive machine-type communications, such as sensor networks, smart metering, 'internet-of-things', or process and factory automation, are forecast to have great economic impact in the next five to ten years. Low-complexity…

It has been shown that an extension of the basic binary polar transformation also polarizes over finite fields. With it the direct encoding of q-ary sources and channels is a process that can be implemented with simple and efficient…

Although LLM inference has emerged as a critical workload for many downstream applications, efficiently inferring LLMs is challenging due to the substantial memory footprint and bandwidth requirements. In parallel, compute capabilities have…

In real world, our datasets often contain outliers. Moreover, the outliers can seriously affect the final machine learning result. Most existing algorithms for handling outliers take high time complexities (e.g. quadratic or cubic…

Quantization emerges as one of the most promising compression technologies for deploying efficient large models for various real time application in recent years. Considering that the storage and IO of weights take up the vast majority of…