M-ary Detection and q-ary Decoding in Large-Scale MIMO: A Non-Binary Belief Propagation Approach

In this paper, we propose a non-binary belief propagation approach (NB-BP) for detection of $M$-ary modulation symbols and decoding of $q$-ary LDPC codes in large-scale multiuser MIMO systems. We first propose a message passing based symbol detection algorithm which computes vector messages using a scalar Gaussian approximation of interference, which results in a total complexity of just $O(KN\sqrt{M})$, where $K$ is the number of uplink users and $N$ is the number of base station (BS) antennas. The proposed NB-BP detector does not need to do a matrix inversion, which gives a complexity advantage over MMSE detection. We then design optimized $q$-ary LDPC codes by matching the EXIT charts of the proposed detector and the LDPC decoder. Simulation results show that the proposed NB-BP detection-decoding approach using the optimized LDPC codes achieve significantly better performance (by about 1 dB to 7 dB at $10^{-5}$ coded BER for various system loading factors with number of users ranging from 16 to 128 and number of BS antennas fixed at 128) compared to using linear detectors (e.g., MMSE detector) and off-the-shelf $q$-ary irregular LDPC codes. Also, even with estimated channel knowledge (e.g., with MMSE channel estimate), the performance of the proposed NB-BP detector is better than that of the MMSE detector.