Related papers: Turbo codes: the phase transition
The reliability function of memoryless channels with noiseless feedback and variable-length coding has been found to be a linear function of the average rate in the classic work of Burnashev. In this work we consider unifilar channels with…
Linear Non-Linear(LN) models are widely used to characterize the receptive fields of early-stage auditory processing. We apply the principle of efficient coding to the LN model of Spectro-Temporal Receptive Fields(STRFs) of the neurons in…
Transformers process tokens in parallel but are temporally shallow: at position $t$, each layer attends to key-value pairs computed based on the previous layer, yielding a depth capped by the number of layers. Recurrent models offer…
This paper explores the design of convolutional codes for varying constraint lengths, focusing on their role in error correction in digital communication systems. Convolutional codes are essential in achieving reliable data transmission…
Many satellite communication systems operating today employ low cost upconverters or downconverters which create phase noise. This noise can severely limit the information rate of the system and pose a serious challenge for the detection…
The lack of uniqueness arising by oversampling of Fourier coefficients is shown to provide a way of transmitting hidden information. A basic encoding/decoding system, developed on the basis of such a possibility, is discussed. The system is…
Coherent information quantifies the transmittable quantum information through a channel and is directly linked to the channel's quantum capacity. In a monitored quantum circuit, regarded as a quantum channel, extensive and positive coherent…
We present a novel information-theoretic framework, termed as TURBO, designed to systematically analyse and generalise auto-encoding methods. We start by examining the principles of information bottleneck and bottleneck-based networks in…
We introduce a novel type of quantum error correcting code, called the spinor code, based on spaces defined by total spin. The code is a nonstabilizer code, and is also a nonlinear quantum error correcting code, meaning that quantum…
Speculative decoding has emerged as an effective approach for accelerating autoregressive inference by parallelizing token generation through a draft-then-verify paradigm. However, existing methods rely on static drafting lengths and rigid…
We study energy propagation in locally time-periodically driven disordered nonlinear chains. For frequencies inside the band of linear Anderson modes, three different regimes are observed with increasing driver amplitude: 1) Below…
We consider asynchronous communication over point-to-point discrete memoryless channels. The transmitter starts sending one block codeword at an instant that is uniformly distributed within a certain time period, which represents the level…
This work constructs codes that are efficiently decodable from a constant fraction of \emph{worst-case} insertion and deletion errors in three parameter settings: (i) Binary codes with rate approaching 1; (ii) Codes with constant rate for…
We introduce a class of cyclic quantum codes, basing the construction not on the simplicity of the stabilizers, but rather on the simplicity of preparation of a code state (at least in the absence of noise). We show how certain known codes,…
Mapping quantum error correcting codes to classical disordered statistical mechanics models and studying the phase diagram of the latter has proven a powerful tool to study the fundamental error robustness and associated critical error…
Spread codes and orbit codes are special families of constant dimension subspace codes. These codes have been well-studied for their error correction capability and transmission rate, but the question of how to encode messages has not been…
In this paper we propose a new efficient message passing algorithm for decoding LDPC transmitted over a channel with strong phase noise. The algorithm performs approximate bayesian inference on a factor graph representation of the channel…
We consider the problem of computing a binary linear transformation using unreliable components when all circuit components are unreliable. Two noise models of unreliable components are considered: probabilistic errors and permanent errors.…
In this paper, we introduce DICOD, a convolutional sparse coding algorithm which builds shift invariant representations for long signals. This algorithm is designed to run in a distributed setting, with local message passing, making it…
We use a quantum formalism for the partition function of the classical $XY$ model to identify a resilience phase transition in a noisy toric-rotor code. Specifically, we consider the toric-rotor code under phase-shift noise described by a…