File Download
There are no files associated with this item.
Links for fulltext
(May Require Subscription)
- Publisher Website: 10.1109/JSAC.2022.3192053
- Scopus: eid_2-s2.0-85135748311
- Find via
Supplementary
-
Citations:
- Scopus: 0
- Appears in Collections:
Article: Hybrid Reinforcement Learning for STAR-RISs: A Coupled Phase-Shift Model Based Beamformer
Title | Hybrid Reinforcement Learning for STAR-RISs: A Coupled Phase-Shift Model Based Beamformer |
---|---|
Authors | |
Keywords | Beamforming deep reinforcement learning (DRL) reconfigurable intelligent surfaces (RISs) simultaneous transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) |
Issue Date | 2022 |
Citation | IEEE Journal on Selected Areas in Communications, 2022, v. 40, n. 9, p. 2556-2569 How to Cite? |
Abstract | A simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) assisted multi-user downlink multiple-input single-output (MISO) communication system is investigated. In contrast to the existing ideal STAR-RIS model assuming an independent transmission and reflection phase-shift control, a practical coupled phase-shift model is considered. Then, a joint active and passive beamforming optimization problem is formulated for minimizing the long-term transmission power consumption, subject to the coupled phase-shift constraint and the minimum data rate constraint. Despite the coupled nature of the phase-shift model, the formulated problem is solved by invoking a hybrid continuous and discrete phase-shift control policy. Inspired by this observation, a pair of hybrid reinforcement learning (RL) algorithms, namely the hybrid deep deterministic policy gradient (hybrid DDPG) algorithm and the joint DDPG & deep-Q network (DDPG-DQN) based algorithm are proposed. The hybrid DDPG algorithm controls the associated high-dimensional continuous and discrete actions by relying on the hybrid action mapping. By contrast, the joint DDPG-DQN algorithm constructs two Markov decision processes (MDPs) relying on an inner and an outer environment, thereby amalgamating the two agents to accomplish a joint hybrid control. Simulation results demonstrate that the STAR-RIS has superiority over other conventional RISs in terms of its energy consumption. Furthermore, both the proposed algorithms outperform the baseline DDPG algorithm, and the joint DDPG-DQN algorithm achieves a superior performance, albeit at an increased computational complexity. |
Persistent Identifier | http://hdl.handle.net/10722/349770 |
ISSN | 2023 Impact Factor: 13.8 2023 SCImago Journal Rankings: 8.707 |
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Zhong, Ruikang | - |
dc.contributor.author | Liu, Yuanwei | - |
dc.contributor.author | Mu, Xidong | - |
dc.contributor.author | Chen, Yue | - |
dc.contributor.author | Wang, Xianbin | - |
dc.contributor.author | Hanzo, Lajos | - |
dc.date.accessioned | 2024-10-17T07:00:42Z | - |
dc.date.available | 2024-10-17T07:00:42Z | - |
dc.date.issued | 2022 | - |
dc.identifier.citation | IEEE Journal on Selected Areas in Communications, 2022, v. 40, n. 9, p. 2556-2569 | - |
dc.identifier.issn | 0733-8716 | - |
dc.identifier.uri | http://hdl.handle.net/10722/349770 | - |
dc.description.abstract | A simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) assisted multi-user downlink multiple-input single-output (MISO) communication system is investigated. In contrast to the existing ideal STAR-RIS model assuming an independent transmission and reflection phase-shift control, a practical coupled phase-shift model is considered. Then, a joint active and passive beamforming optimization problem is formulated for minimizing the long-term transmission power consumption, subject to the coupled phase-shift constraint and the minimum data rate constraint. Despite the coupled nature of the phase-shift model, the formulated problem is solved by invoking a hybrid continuous and discrete phase-shift control policy. Inspired by this observation, a pair of hybrid reinforcement learning (RL) algorithms, namely the hybrid deep deterministic policy gradient (hybrid DDPG) algorithm and the joint DDPG & deep-Q network (DDPG-DQN) based algorithm are proposed. The hybrid DDPG algorithm controls the associated high-dimensional continuous and discrete actions by relying on the hybrid action mapping. By contrast, the joint DDPG-DQN algorithm constructs two Markov decision processes (MDPs) relying on an inner and an outer environment, thereby amalgamating the two agents to accomplish a joint hybrid control. Simulation results demonstrate that the STAR-RIS has superiority over other conventional RISs in terms of its energy consumption. Furthermore, both the proposed algorithms outperform the baseline DDPG algorithm, and the joint DDPG-DQN algorithm achieves a superior performance, albeit at an increased computational complexity. | - |
dc.language | eng | - |
dc.relation.ispartof | IEEE Journal on Selected Areas in Communications | - |
dc.subject | Beamforming | - |
dc.subject | deep reinforcement learning (DRL) | - |
dc.subject | reconfigurable intelligent surfaces (RISs) | - |
dc.subject | simultaneous transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) | - |
dc.title | Hybrid Reinforcement Learning for STAR-RISs: A Coupled Phase-Shift Model Based Beamformer | - |
dc.type | Article | - |
dc.description.nature | link_to_subscribed_fulltext | - |
dc.identifier.doi | 10.1109/JSAC.2022.3192053 | - |
dc.identifier.scopus | eid_2-s2.0-85135748311 | - |
dc.identifier.volume | 40 | - |
dc.identifier.issue | 9 | - |
dc.identifier.spage | 2556 | - |
dc.identifier.epage | 2569 | - |
dc.identifier.eissn | 1558-0008 | - |