Intelligent Navigation of a Magnetic Microrobot with Model-Free Deep Reinforcement Learning in a Real-World Environment

Abstract

Microrobotics has opened new horizons for various applications, especially in medicine. However, it also witnessed challenges in achieving maximum optimal performance. One key challenge is the intelligent, autonomous, and precise navigation control of microrobots in fluid environments. The intelligence and autonomy in microrobot control, without the need for prior knowledge of the entire system, can offer significant opportunities in scenarios where their models are unavailable. In this study, two control systems based on model-free deep reinforcement learning were implemented to control the movement of a disk-shaped magnetic microrobot in a real-world environment. The training and results of an off-policy SAC algorithm and an on-policy TRPO algorithm revealed that the microrobot successfully learned the optimal path to reach random target positions. During training, the TRPO exhibited a higher sample efficiency and greater stability. The TRPO and SAC showed 100% and 97.5% success rates in reaching the targets in the evaluation phase, respectively. These findings offer basic insights into achieving intelligent and autonomous navigation control for microrobots to advance their capabilities for various applications.