Related papers: Information Theoretic Bounds Based Channel Quantiz…
Transformer-based architectures have become the de-facto standard models for a wide range of Natural Language Processing tasks. However, their memory footprint and high latency are prohibitive for efficient deployment and inference on…
We introduce conferencing-based distributed channel quantizers for two-user interference networks where interference signals are treated as noise. Compared with the conventional distributed quantizers where each receiver quantizes its own…
There has been many papers in academic literature on quantizing weight tensors in deep learning models to reduce inference latency and memory footprint. TVM also has the ability to quantize weights and support low-bit computations. Although…
The quantum capacity of a memoryless channel is often used as a single figure of merit to characterize its ability to transmit quantum information coherently. The capacity determines the maximal rate at which we can code reliably over…
In this paper we study quantum communication channels with correlated noise effects, i.e., quantum channels with memory. We derive a model for correlated noise channels that includes a channel memory state. We examine the case where the…
Quantization-aware training (QAT) is an effective method to drastically reduce the memory footprint of LLMs while keeping performance degradation at an acceptable level. However, the optimal choice of quantization format and bit-width…
This paper presents a novel end-to-end methodology for enabling the deployment of low-error deep networks on microcontrollers. To fit the memory and computational limitations of resource-constrained edge-devices, we exploit mixed…
We study optimal rates for quantum communication over a single use of a channel, which itself can correspond to a finite number of uses of a channel with arbitrarily correlated noise. The corresponding capacity is often referred to as the…
Flash memory-based processing-in-memory (flash-based PIM) offers high storage capacity and computational efficiency but faces significant reliability challenges due to noise in high-density multi-level cell (MLC) flash memories. Existing…
One of the main figures of merit for quantum memories and quantum communication devices is their quantum capacity. It has been studied for arbitrary kinds of quantum channels, but its practical estimation has so far been limited to devices…
We consider a continuous-time bandlimited additive white Gaussian noise channel with 1-bit output quantization. On such a channel the information is carried by the temporal distances of the zero-crossings of the transmit signal. The set of…
We present a general model for quantum channels with memory, and show that it is sufficiently general to encompass all causal automata: any quantum process in which outputs up to some time t do not depend on inputs at times t' > t can be…
We consider the one-bit quantizer that minimizes the mean squared error for a source living in a real Hilbert space. The optimal quantizer is a projection followed by a thresholding operation, and we provide methods for identifying the…
In this letter, we study the reference signal-aided channel estimation concept which is a crucial requirement to address the realistic performance of spatial media-based modulation (SMBM) systems where the radio frequency mirrors are…
We consider the problem of estimating an upper bound on the capacity of a memoryless channel with unknown channel law and continuous output alphabet. A novel data-driven algorithm is proposed that exploits the dual representation of…
Large Language Models (LLMs) deliver strong performance across a wide range of NLP tasks, but their massive sizes hinder deployment on resource-constrained devices. To reduce their computational and memory burden, various compression…
Recent machine learning methods use increasingly large deep neural networks to achieve state of the art results in various tasks. The gains in performance come at the cost of a substantial increase in computation and storage requirements.…
Efficient deployment of Large Language Models (LLMs) requires batching multiple requests together to improve throughput. As the batch size, context length, or model size increases, the size of the key and value (KV) cache can quickly become…
Quantization is essential for reducing the computational cost and memory usage of deep neural networks, enabling efficient inference on low-precision hardware. Despite the growing adoption of uniform and floating-point quantization schemes,…
We calculate the quantum capacity of an amplitude-damping channel with time correlated Markov noise, for two channel uses. Our results show that memory of the channel increases it's ability to transmit quantum information significantly. We…