Related papers: A Novel Multi-Task Learning Empowered Codebook Des…

Sparse Mixture of Experts (SMoE) enables efficient training of large language models by routing input tokens to a select number of experts. However, training SMoE remains challenging due to the issue of representation collapse. Recent…

In this paper, pattern division multiple access with large-scale antenna array (LSA-PDMA) is proposed as a novel non-orthogonal multiple access (NOMA) scheme. In the proposed scheme, pattern is designed in both beam domain and power domain…

We consider communication over the Gaussian multiple-access channel in the regime where the number of users grows linearly with the codelength. In this regime, schemes based on sparse superposition coding can achieve a near-optimal tradeoff…

Sparse Autoencoders (SAEs) have emerged as a powerful unsupervised method for extracting sparse representations from language models, yet scalable training remains a significant challenge. We introduce a suite of 256 SAEs, trained on each…

Non-Orthogonal Multiple Access (NOMA) technology has emerged as a promising technology to enable massive connectivity and enhanced spectral efficiency in next-generation wireless networks. In this study, we propose a novel two-user downlink…

Sparse coding is a class of unsupervised methods for learning a sparse representation of the input data in the form of a linear combination of a dictionary and a sparse code. This learning framework has led to state-of-the-art results in…

Currently, progressively larger deep neural networks are trained on ever growing data corpora. As this trend is only going to increase in the future, distributed training schemes are becoming increasingly relevant. A major issue in…

Non-orthogonal multiple access (NOMA) is envisioned to be one of the most beneficial technologies for next generation wireless networks due to its enhanced performance compared to other conventional radio access techniques. Although the…

Convolutional sparse coding (CSC) improves sparse coding by learning a shift-invariant dictionary from the data. However, existing CSC algorithms operate in the batch mode and are expensive, in terms of both space and time, on large…

Sparse coding aims to model data vectors as sparse linear combinations of basis elements, but a majority of related studies are restricted to continuous data without spatial or temporal structure. A new model-based sparse coding (MSC)…

This paper investigates the application of simultaneous wireless information and power transfer (SWIPT) to cooperative non-orthogonal multiple access (NOMA). A new cooperative multiple-input single-output (MISO) SWIPT NOMA protocol is…

Non-orthogonal multiple access (NOMA) has been proposed for massive connectivity in future generations of wireless communications. A dominant NOMA scheme is based on power optimization, in which decoding of target user is assumed to be…

In this paper, we propose an easily trained yet powerful representation learning approach with performance highly competitive to deep neural networks in a digital pathology image segmentation task. The method, called sparse coding driven…

In this paper, we propose a new downlink non-orthogonal multiuser superposition transmission scheme for future 5G cellular networks, which we refer to as the lattice partition multiple access (LPMA). In this proposed design, the base…

Sparse Autoencoders (SAEs) have proven to be powerful tools for interpreting neural networks by decomposing hidden representations into disentangled, interpretable features via sparsity constraints. However, conventional SAEs are…

In this paper, we describe the deep sparse coding network (SCN), a novel deep network that encodes intermediate representations with nonnegative sparse coding. The SCN is built upon a number of cascading bottleneck modules, where each…

Transformer architectures have achieved remarkable success across language, vision, and multimodal tasks, and there is growing demand for them to address in-context compositional learning tasks. In these tasks, models solve the target…

In this paper, we consider the precoder design for downlink multiple-input multiple-output (MIMO) rate-splitting multiple access (RSMA) systems. The proposed scheme with simultaneous diagonalization (SD) decomposes the MIMO channel matrices…

Speculative decoding is a powerful technique for reducing the latency of Large Language Models (LLMs), offering a fault-tolerant framework that enables the use of highly compressed draft models. In this work, we introduce Self-Distilled…

We propose an efficient algorithm for the generalized sparse coding (SC) inference problem. The proposed framework applies to both the single dictionary setting, where each data point is represented as a sparse combination of the columns of…