Related papers: Matched Metrics and Channels
Decoding sits between a language model and everything we do with it, yet it is still treated as a heuristic knob-tuning exercise. We argue decoding should be understood as a principled optimisation layer: at each token, we solve a…
We compare the performance of short-length linear binary codes on the binary erasure channel and the binary-input Gaussian channel. We use a universal decoder that can decode any linear binary block code: Gaussian-elimination based…
We consider probabilistic shaping to maximize the achievable information rate of coded modulation (CM) with hard decision decoding. The proposed scheme using binary staircase codes outperforms its uniform CM counterpart by more than 1.3 dB…
In this paper, we first introduce the concept of elementary linear subspace, which has similar properties to those of a set of coordinates. We then use elementary linear subspaces to derive properties of maximum rank distance (MRD) codes…
We derive an upper bound on the reliability function of mismatched decoding for zero-rate codes. The bound is based on a result by Koml\'os that shows the existence of a subcode with certain symmetry properties. The bound is shown to…
In contrast to a maximum-likelihood decoder, it is often desirable to use an incomplete decoder that can detect its decoding errors with high probability. One common choice is the bounded distance decoder. Bounds are derived for the total…
We provide two results concerning the optimality of the maximum mutual information (MMI) decoder. First, we prove that the error exponents of the typical random codes under the optimal maximum likelihood (ML) decoder and the MMI decoder are…
We consider the problem of erasure/list decoding using certain classes of simplified decoders. Specifically, we assume a class of erasure/list decoders, such that a codeword is in the list if its likelihood is larger than a threshold. This…
Mathematical programming is a branch of applied mathematics and has recently been used to derive new decoding approaches, challenging established but often heuristic algorithms based on iterative message passing. Concepts from mathematical…
We extend the notion of list decoding to {\em ratio list decoding} which involves a list decoder whose list size is specified as a function of the number of messages $M_n$ and the block length $n$. We present necessary and sufficient…
This paper studies the mismatched decoding problem for binary-input discrete memoryless channels. An example is provided for which an achievable rate based on superposition coding exceeds the LM rate (Hui, 1983; Csisz\'ar-K\"orner, 1981),…
We consider the topic of universal decoding with a decoder that does not have direct access to the codebook, but only to noisy versions of the various randomly generated codewords, a problem motivated by biometrical identification systems.…
The min-sum approximation is widely used in the decoding of polar codes. Although it is a numerical approximation, hardly any penalties are incurred in practice. We give a theoretical justification for this. We consider the common case of a…
We propose the first non-trivial generic decoding algorithm for codes in the sum-rank metric. The new method combines ideas of well-known generic decoders in the Hamming and rank metric. For the same code parameters and number of errors,…
Motivated by a greedy approach for generating {\it{information stable}} processes, we prove a universal maximum likelihood (ML) upper bound on the capacities of discrete information stable channels, including the binary erasure channel…
This dissertation focuses on fountain codes under maximum likelihood (ML) decoding. First LT codes are considered under a practical and widely used ML decoding algorithm known as inactivation decoding. Different analysis techniques are…
Upper and lower bounds on the error probability of linear codes under maximum-likelihood (ML) decoding are shortly surveyed and applied to ensembles of codes on graphs. For upper bounds, focus is put on Gallager bounding techniques and…
Embedding-based similarity metrics between text sequences can be influenced not just by the content dimensions we most care about, but can also be biased by spurious attributes like the text's source or language. These document confounders…
A packing lemma is proved using a setting where the channel is a binary-input discrete memoryless channel $(\mathcal{X},w(y|x),\mathcal{Y})$, the code is selected at random subject to parity-check constraints, and the decoder is a joint…
A group of transition probability functions form a Shannon's channel whereas a group of truth functions form a semantic channel. Label learning is to let semantic channels match Shannon's channels and label selection is to let Shannon's…