相关论文: Maxwell Construction: The Hidden Bridge between It…
There is a fundamental relationship between belief propagation and maximum a posteriori decoding. The case of transmission over the binary erasure channel was investigated in detail in a companion paper. This paper investigates the…
We consider communication over memoryless channels using low-density parity-check code ensembles above the iterative (belief propagation) threshold. What is the computational complexity of decoding (i.e., of reconstructing all the typical…
The posterior matching scheme, for feedback encoding of a message point lying on the unit interval over memoryless channels, maximizes mutual information for an arbitrary number of channel uses. However, it in general does not always…
A multilayer network is composed of multiple layers, where different layers have the same set of vertices but represent different types of interactions. Nevertheless, some layers are interdependent or structurally similar in the multilayer…
This work builds on our earlier work on designing demodulators for diffusion-based molecular communications using a Markovian approach. The demodulation filters take the form of an ordinary differential equation (ODE) which computes the…
We are assisting at a growing interest in the development of learning architectures with application to digital communication systems. Herein, we consider the detection/decoding problem. We aim at developing an optimal neural architecture…
Maximum A Posteriori inference in graphical models is often solved via message-passing algorithms, such as the junction-tree algorithm, or loopy belief-propagation. The exact solution to this problem is well known to be exponential in the…
Optimal reconstruction of a source sequence from multiple noisy traces corrupted by random insertions, deletions, and substitutions typically requires joint processing of all traces, leading to computational complexity that grows…
An emerging solution for explaining Transformer-based models is to use vector-based analysis on how the representations are formed. However, providing a faithful vector-based explanation for a multi-layer model could be challenging in three…
Spatially coupled codes have been of interest recently owing to their superior performance over memoryless binary-input channels. The performance is good both asymptotically, since the belief propagation thresholds approach capacity, as…
Information transmission over channels with transceiver distortion is investigated via generalized mutual information (GMI) under Gaussian input distribution and nearest-neighbor decoding. A canonical transceiver structure in which the…
Belief propagation is a powerful tool in statistical physics, machine learning, and modern coding theory. As a decoding method, it is ubiquitous in classical error correction and has also been applied to stabilizer-based quantum error…
Transformer-based neural decoders have emerged as a promising approach to error correction coding, combining data-driven adaptability with efficient modeling of long-range dependencies. This paper presents a novel decoder architecture that…
Deep unfolding is a method of growing popularity that fuses iterative optimization algorithms with tools from neural networks to efficiently solve a range of tasks in machine learning, signal and image processing, and communication systems.…
Max-product "belief propagation" is an iterative, local, message-passing algorithm for finding the maximum a posteriori (MAP) assignment of a discrete probability distribution specified by a graphical model. Despite the spectacular success…
This paper provides a dual domain derivation of the error exponent of maximum mutual information (MMI) decoding with constant composition codes, showing it coincides with that of maximum likelihood decoding for discrete memoryless channels.…
We study a hidden Markov process which is the result of a transmission of the binary symmetric Markov source over the memoryless binary symmetric channel. This process has been studied extensively in Information Theory and is often used as…
Message passing, also known as belief propagation, is a versatile framework for analyzing models defined on graphs. Its most prototypical application is percolation; yet, the interpretation of the message passing formulation of percolation…
What computational structure are we building into large language models when we train them on next-token prediction? Here, we present evidence that this structure is given by the meta-dynamics of belief updating over hidden states of the…
It was recently shown that the problem of decoding messages transmitted through a noisy channel can be formulated as a belief updating task over a probabilistic network [McEliece]. Moreover, it was observed that iterative application of the…