Related papers: Constant Composition Distribution Matching
A distribution matcher (DM) encodes a binary input data sequence into a sequence of symbols (codeword) with desired target probability distribution. The set of the output codewords constitutes a codebook (or code) of a DM.…
Distribution matching is the process of invertibly mapping a uniformly distributed input sequence onto sequences that approximate the output of a desired discrete memoryless source. The special case of a binary output alphabet and…
In this work, binary block-to-block distribution matching is considered. m independent and uniformly distributed bits are mapped to n output bits resembling a target product distribution. A rate R is called achieved by a sequence of…
The number of random bits required to approximate a target distribution in terms of un-normalized informational divergence is considered. It is shown that for a variable-to-variable length encoder, this number is lower bounded by the…
Several applications in communication, control, and learning require approximating target distributions to within small informational divergence (I-divergence). The additional requirement of invertibility usually leads to using encoders…
Distribution matching (DM) transforms independent and Bernoulli(1/2) distributed bits into a sequence of output symbols with a desired distribution. A fixed-to-fixed length, invertible DM architecture based on shell mapping is presented. It…
Distribution matchers for finite alphabets are shown to have informational divergences that grow logarithmically with the block length, generalizing a basic result for binary strings.
Consider Bernoulli(1/2) percolation on $\mathbb{Z}^d$, and define a perfect matching between open and closed vertices in a way that is a deterministic equivariant function of the configuration. We want to find such matching rules that make…
Semantic communication systems often use an end-to-end neural network to map input data into continuous symbols. These symbols, which are essentially neural network features, usually have fixed dimensions and heavy-tailed distributions.…
We consider an ensemble of constant composition codes that are subsets of linear codes: while the encoder uses only the constant-composition subcode, the decoder operates as if the full linear code was used, with the motivation of…
In this paper, we propose a class of linear codes and obtain their weight distribution. Some of these codes are almost optimal. Moreover, several classes of constant composition codes(CCCs) are constructed as subcodes of linear codes.
A constant-rate encoder--decoder pair is presented for a fairly large family of two-dimensional (2-D) constraints. Encoding and decoding is done in a row-by-row manner, and is sliding-block decodable. Essentially, the 2-D constraint is…
Consensus is a common method for computing a function of the data distributed among the nodes of a network. Of particular interest is distributed average consensus, whereby the nodes iteratively compute the sample average of the data stored…
We introduce the notion of combinatorial encoding of continuous dynamical systems and suggest the first examples, which are the most interesting and important, namely, the combinatorial encoding of a Bernoulli process with continuous state…
The original purpose of component-based development was to provide techniques to master complex software, through composition, reuse and parametrisation. However, such systems are rapidly moving towards a level in which software becomes…
Without imposing prior distributional knowledge underlying multivariate time series of interest, we propose a nonparametric change-point detection approach to estimate the number of change points and their locations along the temporal axis.…
Consider a distributed coding for computing problem with constant decoding locality, i.e., with a vanishing error probability, any single sample of the function can be approximately recovered by probing only constant number of compressed…
A composition of a nonnegative integer (n) is a sequence of positive integers whose sum is (n). A composition is palindromic if it is unchanged when its terms are read in reverse order. We provide a generating function for the number of…
We investigate the task of retrieving information from compositional distributed representations formed by Hyperdimensional Computing/Vector Symbolic Architectures and present novel techniques which achieve new information rate bounds.…
We study Transformer overparameterization through the lens of angular similarity in high-dimensional encoder-decoder embeddings. We apply Bernoulli dropout between the encoder and the decoder, varying the keep probability $p$ to identify a…