Related papers: The Redundancy of Slepian-Wolf Coding Revisited
Words in natural language follow a Zipfian distribution whereby some words are frequent but most are rare. Learning representations for words in the "long tail" of this distribution requires enormous amounts of data. Representations of rare…
We consider a wireless sensors network scenario where two nodes detect correlated sources and deliver them to a central collector via a wireless link. Differently from the Slepian-Wolf approach to distributed source coding, in the proposed…
We analyze the performance of a linear code used for a data compression of Slepian-Wolf type. In our framework, two correlated data are separately compressed into codewords employing Gallager-type codes and casted into a communication…
Despite strong performance on a variety of tasks, neural sequence models trained with maximum likelihood have been shown to exhibit issues such as length bias and degenerate repetition. We study the related issue of receiving…
In this paper we provide new compact integral expressions and associated simple asymptotic approximations for converse and achievability bounds in the finite blocklength regime. The chosen converse and random coding union bounds were taken…
The order of letters is not always relevant in a communication task. This paper discusses the implications of order irrelevance on source coding, presenting results in several major branches of source coding theory: lossless coding,…
This document is written in order to establish a common base ground on which the majority of the relevant research about linear fountain codes can be analyzed and compared. As far as I am concerned, there is no unified approach that…
Ultra-Reliable Low-Latency Communications have stringent delay constraints, and hence use codes with small block length (short codewords). In these cases, classical models that provide good approximations to systems with infinitely long…
In this paper, we investigate the redundancy of universal coding schemes on smooth parametric sources in the finite-length regime. We derive an upper bound on the probability of the event that a sequence of length $n$, chosen using…
This paper presents new lower and upper bounds for the optimal compression of binary prefix codes in terms of the most probable input symbol, where compression efficiency is determined by the nonlinear codeword length objective of…
Scaling the amount of compute used to train language models has dramatically improved their capabilities. However, when it comes to inference, we often limit models to making only one attempt at a problem. Here, we explore inference compute…
Cosine similarity of contextual embeddings is used in many NLP tasks (e.g., QA, IR, MT) and metrics (e.g., BERTScore). Here, we uncover systematic ways in which word similarities estimated by cosine over BERT embeddings are understated and…
We consider the problem of distributed compression for correlated quantum sources. The classical version of this problem was solved by Slepian and Wolf, who showed that distributed compression could take full advantage of redundancy in the…
In distributed multilevel diversity coding, $K$ correlated sources (each with $K$ components) are encoded in a distributed manner such that, given the outputs from any $\alpha$ encoders, the decoder can reconstruct the first $\alpha$…
Motivated by streaming multi-view video coding and wireless sensor networks, we consider the problem of blockwise streaming compression of a pair of correlated sources, which we term streaming Slepian-Wolf coding. We study the moderate…
Redundancy for straggler mitigation, originally in data download and more recently in distributed computing context, has been shown to be effective both in theory and practice. Analysis of systems with redundancy has drawn significant…
Rank-order coding, a form of temporal coding, has emerged as a promising scheme to explain the rapid ability of the mammalian brain. Owing to its speed as well as efficiency, rank-order coding is increasingly gaining interest in diverse…
This paper describes in detail how erasure codes are implemented in the Swarm system. First, in Section 1, we introduce erasure codes, and show how to apply them to files in Swarm (Section 2). In Section 3, we introduce security levels of…
A new run length encoding algorithm for lossless data compression that exploits positional redundancy by representing data in a two-dimensional model of concentric circles is presented. This visual transform enables detection of runs (each…
In this paper we consider point-to-point and distributed source coding problems where the receiver is only interested in a function of the data sent by the source encoder(s), while knowledge of the function remains unknown to the…