Theoretical and Experimental Investigation of Free-Floating Space Manipulator Motion Control Using Reinforcement Learning

This doctoral research is motivated by the need to capture a non-cooperative target in Earth orbits by autonomous manipulators for debris removal. It investigates the critical motion planning problem for a 6DOF free-floating space manipulator using model-free Reinforcement Learning. This problem is caused by dynamic coupling between the spacecraft and robotic manipulator, which significantly affects control and precision in the space environment. This research addresses critical requirements for efficient and effective manipulation in space, including accurate pose alignment between the end-effector and target debris, collision avoidance with both the target and other links of the manipulator and external obstacles, smoothing of joint velocities using optimization terms integrated into the reward function, and adaptation to the high mass ratio between the manipulator and its base spacecraft. Recognizing the imperfection of real-world sensors, this research also incorporates observation noise in the training process to enhance the agent's resilience to noise.

Additionally, the reinforcement learning agent is trained with varying initial conditions that are randomly set at the start of each episode, including the manipulator’s initial joint angles, target positions, and obstacle locations. The trained agent can find suitable paths for any target location and avoid obstacles regardless of the manipulator's initial position. The study provides a solid foundation for the application of reinforcement learning in complex free-floating space robotic operations and offers insights for future missions.

In addition to numerical verification, ground experimental validation is essential. To overcome the difficulty in mimicking the 3D microgravity environment on Earth, this doctoral research has devoted efforts to designing and building a hardware-in-the-loop ground testbed with active gravity compensation in our lab. The testbed involves two industrial 6DOF robotic arms, one robotic finger gripper and various sensing equipment, including cameras and force/torque sensors, to mimic the 6DOF motion maneuvers that a free-floating robotic manipulator or tumbling satellite/target in microgravity. Initial experimental results in motion control, computer vision, and sensing capabilities are presented to show the potential of the testbed. This facility will be an invaluable tool for the future development and validation of space robotic manipulators, ultimately improving their effectiveness and reliability in space missions.