Related papers: Invertible Low-Divergence Coding
An index coding problem arises when there is a single source with a number of messages and multiple receivers each wanting a subset of messages and knowing a different set of messages a priori. The noiseless Index Coding Problem is to…
Coded computation is a framework which provides redundancy in distributed computing systems to speed up largescale tasks. Although most existing works assume an error-free scenarios in a master-worker setup, the link failures are common in…
In a large class of statistical inverse problems it is necessary to suppose that the transformation that is inverted is known. Although, in many applications, it is unrealistic to make this assumption, the problem is often insoluble without…
Pre-trained Transformer language models (LM) have become go-to text representation encoders. Prior research fine-tunes deep LMs to encode text sequences such as sentences and passages into single dense vector representations for efficient…
Fifth generation new radio aims to facilitate new use cases in wireless communications. Some of these new use cases have highly de-manding latency requirements; many of the powerful forward error correction codes deployed in current…
Given a pair of predictor variables and a response variable, how much information do the predictors have about the response, and how is this information distributed between unique, redundant, and synergistic components? Recent work has…
Convolutional codes are constructed, designed and analysed using row and/or block structures of unit algebraic schemes. Infinite series of such codes and of codes with specific properties are derived. Properties are shown algebraically and…
We consider the classic joint source-channel coding problem of transmitting a memoryless source over a memoryless channel. The focus of this work is on the long-standing open problem of finding the rate of convergence of the smallest…
We develop upper bounds on code size for an independent and identically distributed deletion and insertion channels for a given code length and target frame error probability. The bounds are obtained as a variation of a general converse…
This letter introduces a novel channel coding design framework for short-length codewords that permits balancing the tradeoff between the bit error rate floor and waterfall region by modifying a single real-valued parameter. The proposed…
Consider two remote nodes (encoder and decoder), each with a binary sequence. The encoder's sequence $X$ differs from the decoder's sequence $Y$ by a small number of edits (deletions and insertions). The goal is to construct a message $M$,…
Multi-objective integer or mixed-integer programming problems typically have disconnected feasible domains, making the task of constructing an approximation of the Pareto front challenging. The present paper shows that certain algorithms…
This work develops problem statements related to encoders and autoencoders with the goal of elucidating variational formulations and establishing clear connections to information-theoretic concepts. Specifically, four problems with varying…
Safety-critical autonomous systems must satisfy hard state constraints under tight computational and sensing budgets, yet learning-based controllers are often far more complex than safe operation requires. To formalize this gap, we study…
Reranker models aim to re-rank the passages based on the semantics similarity between the given query and passages, which have recently received more attention due to the wide application of the Retrieval-Augmented Generation. Most previous…
This paper investigates a unification of distributed source coding, multiple description coding, and source coding with side information at decoders. The equivalence between the multiple-decoder extension of distributed source coding with…
Algebraic methods for the design of series of maximum distance separable (MDS) linear block and convolutional codes to required specifications and types are presented. Algorithms are given to design codes to required rate and required…
Modern large language models (LLMs) excel at tasks that require storing and retrieving knowledge, such as factual recall and question answering. Transformers are central to this capability because they can encode information during training…
Computationally intensive distributed and parallel computing is often bottlenecked by a small set of slow workers known as stragglers. In this paper, we utilize the emerging idea of "coded computation" to design a novel…
We consider the asymmetric multilevel diversity (A-MLD) coding problem, where a set of $2^K-1$ information sources, ordered in a decreasing level of importance, is encoded into $K$ messages (or descriptions). There are $2^K-1$ decoders,…