Design and motion control of multi-segment continuum robots

Abstract

Conventional rigid-link robots have been widely deployed in operations involving production, warehousing, material handling, and many other industrial automation applications, but the rigid structures often limit their applicability in complex operating environments such as rescue scenes, underground and underwater spaces. Therefore, inspired by snakes and biological tentacles, continuum robots with soft and continuous backbones are developed where multi-segment continuum robots have superior dexterity for the manipulation in deep and confined spaces. However, the continuous structures are accompanied by very high degrees of freedom with high non-linearity, making their design and motion control extremely challenging. The focus of this thesis is to optimize their design through analyzing their kinematic performance and achieve their motion control by using deep reinforcement learning techniques, which makes important contributions in the following two aspects. Firstly, this thesis investigates the kinematic performance of a generic multi-segment continuum robot for its optimal design. Two representative characteristics, namely, reachable workspace and dexterity performance are considered. The former is studied through simulation by traversing the joint space with Monte Carlo Method. For the latter, two indices are proposed, namely, displacement dexterity in operation position and angularity dexterity in changing orientation of the end effector, which are realized by separating Jacobian matrix. Accordingly, a quantitative assessment system for the kinematic performance of continuum robots is established. By considering both the reachable workspace and dexterity performance, an objective function is defined to explore the optimal design, and it is solved by an evolutionary algorithm – Particle Swarm Optimization algorithm. Case studies on two-segment and three-segment continuum robots show the effectiveness of the method for optimizing structure. The proposed method is generic to continuum robots with different number of segments. Secondly, model-free deep reinforcement learning (DRL) methods are developed for general continuous control of multi-segment continuum robots by learning the complex relationships and achieving the motion control. The control issue is described as a customized Markov Decision Process. Then, the deterministic policy gradient algorithm, a combined value-based and policy-based reinforcement learning (RL) method, is adopted to update the optimal control policy directly from experience replay. Meanwhile, deep neural networks (DNNs) are used for the approximation of policies and state-action value functions in high-dimensional and continuous spaces. Moreover, an improved continuous control is introduced to improve the reliability of policy gradient by avoiding overestimate of the value functions. Integrating DNN into RL eliminates the requirement for analytical model of robots when formulating the control policy, which represents a model-free approach. Particularly, both the joint space and task space are continuous in our work. Case studies are conducted on an optimized two-segment continuum robot, and the comparison results of the motion performance demonstrate the superiority of the improved algorithm in terms of accuracy and generality. The effectiveness and potential of our methods are verified in a number of case studies. To sum up, this thesis provides new insights into the kinematic performance evaluation, the optimization of design, and the adoption of DRL-based control techniques in the field of multi-segment continuum robots.