Related papers: Information Theoretic Bounds Based Channel Quantiz…

In recent years Deep Neural Networks (DNNs) have been rapidly developed in various applications, together with increasingly complex architectures. The performance gain of these DNNs generally comes with high computational costs and large…

The Interference Channels (ICs) represent fundamental building blocks of wireless communication networks. Despite considerable progress in network information theory, available capacity results for ICs, specifically those with more than two…

Quantized neural networks are well known for reducing the latency, power consumption, and model size without significant harm to the performance. This makes them highly appropriate for systems with limited resources and low power capacity.…

Transformer-based models, such as BERT, have been widely applied in a wide range of natural language processing tasks. However, one inevitable side effect is that they require massive memory storage and inference cost when deployed in…

While capacities of discrete memoryless channels are well studied, it is still not possible to obtain a closed-form expression for the capacity of an arbitrary discrete memoryless channel. This paper describes an elementary technique based…

Large language models (LLMs) require immense resources for training and inference. Quantization, a technique that reduces the precision of model parameters, offers a promising solution for improving LLM efficiency and sustainability. While…

We study quantization of log-likelihood ratios (LLR) in bit-interleaved coded modulation (BICM) systems in terms of an equivalent discrete channel. We propose to design the quantizer such that the quantizer outputs become equiprobable. We…

We consider the design of identical one-bit probabilistic quantizers for distributed estimation in sensor networks. We assume the parameter-range to be finite and known and use the maximum Cram\'er-Rao Lower Bound (CRB) over the…

Quantization is an effective technique to reduce memory footprint, inference latency, and power consumption of deep learning models. However, existing quantization methods suffer from accuracy degradation compared to full-precision (FP)…

This paper tackles the problem of single-user multiple-input multiple-output communication with 1-bit digital-to-analog and analog-to-digital converters. With the information-theoretic capacity as benchmark, the complementary strategies of…

Two new asynchronous modulation techniques for molecular timing (MT) channels are proposed. One based on modulating information on the time between two consecutive releases of indistinguishable information particles, and one based on using…

Structured State Space models (SSM) have recently emerged as a new class of deep learning models, particularly well-suited for processing long sequences. Their constant memory footprint, in contrast to the linearly scaling memory demands of…

Quantization is essential for Neural Network (NN) compression, reducing model size and computational demands by using lower bit-width data types, though aggressive reduction often hampers accuracy. Mixed Precision (MP) mitigates this…

This paper studies the capacities of input-driven finite-state channels, i.e., channels whose current state is a time-invariant deterministic function of the previous state and the current input. We lower bound the capacity of such a…

Coresets are small, weighted summaries of larger datasets, aiming at providing provable error bounds for machine learning (ML) tasks while significantly reducing the communication and computation costs. To achieve a better trade-off between…

Marton's inner bound is the best known achievable region for a general discrete memoryless broadcast channel. To compute Marton's inner bound one has to solve an optimization problem over a set of joint distributions on the input and…

Quantization for deep neural networks have afforded models for edge devices that use less on-board memory and enable efficient low-power inference. In this paper, we present a comparison of model-parameter driven quantization approaches…

Having lower quantization resolution, has been introduced in the literature, as a solution to reduce the power consumption of massive MIMO and millimeter wave MIMO systems. In this paper, we analyze bit error rate (BER) performance of…

MIMO systems are considered as most promising for wireless communications. However, with an increasing number of radio front ends the corresponding energy consumption and costs become an issue, which can be relieved by the utilization of…

Large language models (LLMs) have shown immense potential across various domains, but their high memory requirements and inference costs remain critical challenges for deployment. Post-training quantization (PTQ) has emerged as a promising…