File Download
Links for fulltext
(May Require Subscription)
- Publisher Website: 10.3390/jmse11020285
- Scopus: eid_2-s2.0-85149101690
- WOS: WOS:000940472200001
- Find via
Supplementary
- Citations:
- Appears in Collections:
Article: Markovian-Jump Reinforcement Learning for Autonomous Underwater Vehicles under Disturbances with Abrupt Changes
Title | Markovian-Jump Reinforcement Learning for Autonomous Underwater Vehicles under Disturbances with Abrupt Changes |
---|---|
Authors | |
Keywords | autonomous underwater vehicles disturbance rejection markovian-jump systems reinforcement learning |
Issue Date | 27-Jan-2023 |
Publisher | MDPI |
Citation | Journal of Marine Science and Engineering, 2023, v. 11, n. 2 How to Cite? |
Abstract | This paper studies the position regulation problems of an Autonomous Underwater Vehicle (AUV) subject to external disturbances that may have abrupt variations due to some events, e.g., water flow hitting nearby underwater structures. The disturbing forces may frequently exceed the actuator capacities, necessitating a constrained optimization of control inputs over a future time horizon. However, the AUV dynamics and the parameters of the disturbance models are unknown. Estimating the Markovian processes of the disturbances is challenging since it is entangled with uncertainties from AUV dynamics. As opposed to a single-Markovian description, this paper formulates the disturbed AUV as an unknown Markovian-Jump Linear System (MJLS) by augmenting the AUV state with the unknown disturbance state. Based on an observer network and an embedded solver, this paper proposes a reinforcement learning approach, Disturbance-Attenuation-net (MDA-net), for attenuating Markovian-jump disturbances and stabilizing the disturbed AUV. MDA-net is trained based on the sensitivity analysis of the optimality conditions and is able to estimate the disturbance and its transition dynamics based on observations of AUV states and control inputs online. Extensive numerical simulations of position regulation problems and preliminary experiments in a tank testbed have shown that the proposed MDA-net outperforms the existing DOB-net and a classical approach, Robust Integral of Sign of Error (RISE). |
Persistent Identifier | http://hdl.handle.net/10722/338598 |
ISSN | 2023 Impact Factor: 2.7 2023 SCImago Journal Rankings: 0.532 |
ISI Accession Number ID |
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Lu, Wenjie | - |
dc.contributor.author | Huang, Yongquan | - |
dc.contributor.author | Hu, Manman | - |
dc.date.accessioned | 2024-03-11T10:30:05Z | - |
dc.date.available | 2024-03-11T10:30:05Z | - |
dc.date.issued | 2023-01-27 | - |
dc.identifier.citation | Journal of Marine Science and Engineering, 2023, v. 11, n. 2 | - |
dc.identifier.issn | 2077-1312 | - |
dc.identifier.uri | http://hdl.handle.net/10722/338598 | - |
dc.description.abstract | <p>This paper studies the position regulation problems of an Autonomous Underwater Vehicle (AUV) subject to external disturbances that may have abrupt variations due to some events, e.g., water flow hitting nearby underwater structures. The disturbing forces may frequently exceed the actuator capacities, necessitating a constrained optimization of control inputs over a future time horizon. However, the AUV dynamics and the parameters of the disturbance models are unknown. Estimating the Markovian processes of the disturbances is challenging since it is entangled with uncertainties from AUV dynamics. As opposed to a single-Markovian description, this paper formulates the disturbed AUV as an unknown Markovian-Jump Linear System (MJLS) by augmenting the AUV state with the unknown disturbance state. Based on an observer network and an embedded solver, this paper proposes a reinforcement learning approach, Disturbance-Attenuation-net (MDA-net), for attenuating Markovian-jump disturbances and stabilizing the disturbed AUV. MDA-net is trained based on the sensitivity analysis of the optimality conditions and is able to estimate the disturbance and its transition dynamics based on observations of AUV states and control inputs online. Extensive numerical simulations of position regulation problems and preliminary experiments in a tank testbed have shown that the proposed MDA-net outperforms the existing DOB-net and a classical approach, Robust Integral of Sign of Error (RISE).<br></p> | - |
dc.language | eng | - |
dc.publisher | MDPI | - |
dc.relation.ispartof | Journal of Marine Science and Engineering | - |
dc.rights | This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. | - |
dc.subject | autonomous underwater vehicles | - |
dc.subject | disturbance rejection | - |
dc.subject | markovian-jump systems | - |
dc.subject | reinforcement learning | - |
dc.title | Markovian-Jump Reinforcement Learning for Autonomous Underwater Vehicles under Disturbances with Abrupt Changes | - |
dc.type | Article | - |
dc.description.nature | published_or_final_version | - |
dc.identifier.doi | 10.3390/jmse11020285 | - |
dc.identifier.scopus | eid_2-s2.0-85149101690 | - |
dc.identifier.volume | 11 | - |
dc.identifier.issue | 2 | - |
dc.identifier.eissn | 2077-1312 | - |
dc.identifier.isi | WOS:000940472200001 | - |
dc.identifier.issnl | 2077-1312 | - |