Related papers: Capacity-achieving sparse superposition codes with…

Learning rich data representations from unlabeled data is a key challenge towards applying deep learning algorithms in downstream tasks. Several variants of variational autoencoders (VAEs) have been proposed to learn compact data…

We consider spatially coupled low-density parity-check codes with finite smoothing parameters. A finite smoothing parameter is important for designing practical codes that are decoded using low-complexity windowed decoders. By optimizing…

Energy-efficient communication using a class of spatial modulation (SM) that encodes the source information entirely in the antenna indices is considered in this paper. The energy-efficient modulation design is formulated as a convex…

Sparse autoencoders (SAEs) have become a central tool for interpreting language models. However, two key SAE analyses that remain difficult to scale are (1) matching semantically similar features across multi-layers and (2) compressing…

Shannon theory models communication as the reliable transfer of symbol sequences, with performance governed by capacity and rate-distortion limits. When both endpoints possess strong predictors -- as in modern large language models and…

Spatially coupled low-density parity-check codes show an outstanding performance under the low-complexity belief propagation (BP) decoding algorithm. They exhibit a peculiar convergence phenomenon above the BP threshold of the underlying…

Convolutional sparse coding (CSC) is an important building block of many computer vision applications ranging from image and video compression to deep learning. We present two contributions to the state of the art in CSC. First, we…

Superposition codes are efficient for the Additive White Gaussian Noise channel. We provide here a replica analysis of the performances of these codes for large signals. We also consider a Bayesian Approximate Message Passing decoder based…

Sparse code multiple access (SCMA) is a promising code-domain non-orthogonal multiple access (NOMA) scheme for the enabling of massive machine-type communication. In SCMA, the design of good sparse codebooks and efficient multiuser decoding…

Semantic maps are increasingly utilized in areas such as robotics, autonomous systems, and extended reality, motivating the investigation of efficient compression methods that preserve structured semantic information. This paper studies…

This paper studies a large random matrix system (LRMS) model involving an arbitrary signal distribution and forward error control (FEC) coding. We establish an area property based on the so-called Turbo approximate message passing…

Multi-sector capacity expansion models play a crucial role in energy planning by providing decision support for policymaking in technology development. To ensure reliable support, these models require high technological, spatial, and…

This paper introduces a new multivariate convolutional sparse coding based on tensor algebra with a general model enforcing both element-wise sparsity and low-rankness of the activations tensors. By using the CP decomposition, this model…

In this paper, we study codes with sparse generator matrices. More specifically, low-density generator matrix (LDGM) codes with a certain constraint on the weight of the columns in the generator matrix are considered. In this paper, it is…

Transformer architectures have achieved remarkable success across language, vision, and multimodal tasks, and there is growing demand for them to address in-context compositional learning tasks. In these tasks, models solve the target…

Efficient signal detectors are rather important yet challenging to achieve satisfactory performance for large-scale communication systems. This paper considers a non-orthogonal sparse code multiple access (SCMA) configuration for…

Low-density parity-check (LDPC) convolutional codes have been shown to exhibit excellent performance under low-complexity belief-propagation decoding [1], [2]. This phenomenon is now termed threshold saturation via spatial coupling. The…

In this paper, we propose new coupled codes constructed by overlapping circular spatially-coupled low-density parity-check (SC-LDPC) codes, which show better asymptotic and finite-length decoding performance compared to the conventional…

In this paper, we describe the deep sparse coding network (SCN), a novel deep network that encodes intermediate representations with nonnegative sparse coding. The SCN is built upon a number of cascading bottleneck modules, where each…

We study the compressed sensing reconstruction problem for a broad class of random, band-diagonal sensing matrices. This construction is inspired by the idea of spatial coupling in coding theory. As demonstrated heuristically and…