Digital Twin-Driven MADRL Approaches for Communication-Computing-Control Co-Optimization

Abstract

The unpredictability of network environments, limited edge resources, and the high complexity of collaborative policies are significantly hindering the development of the Industrial Internet of Things (IIoT). These challenges are particularly pronounced in healthcare, where high-priority, delay-sensitive medical tasks and large-scale personalized services face substantial obstacles. To address these challenges, this paper proposes the Self-Attention Enhanced QMIX with Multi-Pass Multi-Task Execution (SAE-MT-QMIX) algorithm, aimed at optimizing communication and computing resource allocation as well as task offloading strategies. By leveraging Digital Twin (DT) support, the algorithm achieves collaborative optimization of communication, computing, and control within the Internet of Medical Things (IoMT), significantly enhancing the quality of service for massive personalized applications. The algorithm adopts a distributed execution and centralized training framework: the distributed execution component uses the Multi-Pass Multi-Task Deep Q-Network (MPMT-DQN) algorithm to handle the complexity of parameterized action spaces in multi-task scenarios, while the centralized training component employs the Self-Attention Enhanced QMIX (SAE-QMIX) algorithm to dynamically optimize credit assignment across multiple users. Simulation results demonstrate that SAE-MT-QMIX significantly reduces delay and energy consumption compared to baseline methods. It ensures effective optimization of communication, computing, and control in dynamic IoMT, efficiently addressing diverse demands and tasks while enhancing service quality and system adaptability.