Swarm Robotics Navigation Task: A Comparative Study of Reinforcement Learning and Particle Swarm Optimization Methodologies

Abstract

Automatic design methods focus on generating the collective behavior of swarm robotic systems. These methods enable multiple robots to coordinate and execute complex tasks in their environments autonomously. This research paper investigated two prominent methodologies: particle swarm optimization (PSO) and reinforcement learning (RL). A new comparative study was conducted to analyze the performance of a group of mobile robots through extensive experimentation. The objective was to produce navigational collective behavior through unknown environments. These environments differ in complexity ranging from obstacle-free environments to cluttered ones. The core metrics of the comparison include the time efficiency of individual robots and the overall swarm, flexibility in pathfinding, and the ability to generalize solutions for new environments. The obtained results from the Webots simulator with Python controller suggested that RL excels in environments closely aligned with its training conditions. RL achieved a faster completion time and demonstrated superior coordination among individual robots. However, its performance dips when facing untrained scenarios necessitating computationally expensive retraining or structural complexities to enhance adaptability. Conversely, PSO showed commendable consistency in performance. Despite its slower pace, it exhibited robustness in various challenging settings without reconfiguration.