This paper considers an uplink multiuser multiple-input-multiple-output (MU-MIMO) system. In this system, we have presented a sparse-aware minimum-mean-square-error (SA-MMSE) detector which improves an underlying linear detector using the sparsity of a residual error vector (difference from the transmit vector and the detected one by the linear detector). Despite its attractive performance, the conventional SA-MMSE detector is only available for 4-QAM systems. In this paper, we generalize the SA-MMSE detector for a higher-order modulation system in a non-trivial method. This is referred to as generalized SA-MMSE (GSA-MMSE) detector. The key idea of the proposed detector is to exploit the hierarchical structure of a residual error vector. To be specific, the residual error vector can be decomposed into orthogonal sub-error vectors and, leveraging the orthogonality, the sub-error vectors can be decoded using the corresponding SA-MMSE detector in a successive fashion. Via simulation results, we demonstrate that the GSA-MMSE detector significantly outperforms the conventional linear detectors with a comparable complexity.
ACKNOWLEDGMENT This work was supported by the Ajou University Research Fund, and the frame of international cooperation program managed by the National Research Foundation of Korea under Grant 2017K2A9A2A06016102.