Complex-valued transformer for wireless communications

Abstract

Deep learning-based approaches, especially transformer models with attention mechanisms, have demonstrated powerful capabilities for solving complicated problems in the field of wireless communications. Unlike traditional optimization algorithms, they do not require precise mathematical models and offer fast inference times, making them suitable for challenging optimization problems in 6G systems. However, despite the complex-valued nature of signals from wireless communication systems, existing works usually split and stack the real and imaginary parts into fully real-valued inputs and subsequently apply real-valued neural network operations, which fail to capture the inherent structure of complex-valued signals. Conversely, complex-valued neural networks (CVNNs) have received growing attention for their ability to provide a more natural and effective representation of complex-valued data. However, the existing CVNNs for wireless communication tasks generally adopt complex-valued versions of primitive neural network architectures, leading to performance loss.

To fill this research gap, this thesis proposes a fundamental paradigm for complex-valued transformers, including the complex-valued embedding module, encoding module, decoding module and an out projection module. Based on this structure, customized neural networks are designed by integrating the characteristics of different communication tasks. To demonstrate the potential of the complex-valued transformer for wireless communication, this thesis focuses on three applications: (1) a supervised regression problem: channel estimation, (2) a supervised classification problem: activity detection, and (3) unsupervised precoding design in frequency division duplexing (FDD) systems. These three problems cover a range from regression to classification, supervised to unsupervised learning, and specific module design to end-to-end design.

Experimental results demonstrate that the complex-valued transformer customized for the abovementioned three tasks outperforms all neural-networks-based benchmarks, including complex-valued multi-layer perceptron and real-valued models ranging from deep neural networks to transformers with a comparable number of parameters. Such a comparison reveals that the performance gain of the complex-valued transformer is attributed to its ability to fully capture the internal structure of complex-valued signals. Moreover, when trained on the limited data samples, the complex-valued transformer maintains superior test performance than its real-valued counterparts. This thesis underscores the potential of complex-valued transformers in revolutionizing the modeling, processing, and optimization of wireless communication systems, paving a step toward advancing the current and future generations of wireless networks.