Related papers: Information Theoretic Bounds Based Channel Quantiz…

Optimizer states are a major source of memory consumption for training neural networks, limiting the maximum trainable model within given memory budget. Compressing the optimizer states from 32-bit floating points to lower bitwidth is…

Martingale concentration inequalities constitute a powerful mathematical tool in the analysis of problems in a wide variety of fields ranging from probability and statistics to information theory and machine learning. Here we apply…

Recently, transformer has achieved remarkable performance on a variety of computer vision applications. Compared with mainstream convolutional neural networks, vision transformers are often of sophisticated architectures for extracting…

In this paper, we propose an oversampling based low-resolution aware least squares channel estimator for large-scale multiple-antenna systems with 1-bit analog-to-digital converters on each receive antenna. To mitigate the information loss…

This paper introduces a novel low-complexity memoryless linearizer for suppression of distortion in analog frontends. It is based on our recently introduced linearizer which is inspired by neural networks, but with orders-of-magnitude lower…

We extend a low-rate improvement of the random coding bound on the reliability of a classical discrete memoryless channel to its quantum counterpart. The key observation that we make is that the problem of bounding below the error exponent…

We obtain a maximizer for the quantum mutual information for classical information sent over the quantum qubit amplitude damping channel. This is achieved by limiting the ensemble of input states to antipodal states, in the calculation of…

Diffusion models have received wide attention in generation tasks. However, the expensive computation cost prevents the application of diffusion models in resource-constrained scenarios. Quantization emerges as a practical solution that…

Due to the crossbar array architecture, the sneak-path problem severely degrades the data integrity in the resistive random access memory (ReRAM). In this letter, we investigate the channel quantizer design for ReRAM arrays with multiple…

A class of channels is introduced for which there is memory inside blocks of a specified length and no memory across the blocks. The multi-user model is called an information network with in-block memory (NiBM). It is shown that…

The maximum rates for information transmission through noisy quantum channels has primarily been developed for memoryless channels, where the noise on each transmitted state is treated as independent. Many real world communication channels…

Large language models have shown exceptional capabilities in a wide range of tasks, such as text generation and video generation, among others. However, due to their massive parameter count, these models often require substantial storage…

We study the behavior of channel capacity when a one-bit quantizer is employed at the output of the discrete-time average-power-limited Gaussian channel. We focus on the low signal-to-noise ratio regime, where communication at very low…

Processing-in-memory (PIM), an increasingly studied neuromorphic hardware, promises orders of energy and throughput improvements for deep learning inference. Leveraging the massively parallel and efficient analog computing inside memories,…

This paper explores the combination of neural network quantization and entropy coding for memory footprint minimization. Edge deployment of quantized models is hampered by the harsh Pareto frontier of the accuracy-to-bitwidth tradeoff,…

Diffusion models have demonstrated remarkable performance on vision generation tasks. However, the high computational complexity hinders its wide application on edge devices. Quantization has emerged as a promising technique for inference…

Several analog-to-digital conversion methods for bandlimited signals used in applications, such as Sigma Delta quantization schemes, employ coarse quantization coupled with oversampling. The standard mathematical model for the error accrued…

Several information-theoretic studies on channels with output quantization have identified the capacity-achieving input distributions for different fading channels with 1-bit in-phase and quadrature (I/Q) output quantization. However, an…

This paper provides new insight into the classical problem of determining both the capacity of the discrete-time channel with uniform output quantization and the capacity achieving input distribution. It builds on earlier work by Gallager…

Low-resolution devices are promising for systems that demand low energy consumption and low complexity as required in IoT systems. In this study, we propose a novel waveform for bandlimited channels with 1-bit quantization and oversampling…