Model inductive bias enhanced deep reinforcement learning for robot navigation in crowded environments

Abstract

AbstractNavigating mobile robots in crowded environments poses a significant challenge and is essential for the coexistence of robots and humans in future intelligent societies. As a pragmatic data-driven approach, deep reinforcement learning (DRL) holds promise for addressing this challenge. However, current DRL-based navigation methods have possible improvements in understanding agent interactions, feedback mechanism design, and decision foresight in dynamic environments. This paper introduces the model inductive bias enhanced deep reinforcement learning (MIBE-DRL) method, drawing inspiration from a fusion of data-driven and model-driven techniques. MIBE-DRL extensively incorporates model inductive bias into the deep reinforcement learning framework, enhancing the efficiency and safety of robot navigation. The proposed approach entails a multi-interaction network featuring three modules designed to comprehensively understand potential agent interactions in dynamic environments. The pedestrian interaction module can model interactions among humans, while the temporal and spatial interaction modules consider agent interactions in both temporal and spatial dimensions. Additionally, the paper constructs a reward system that fully accounts for the robot’s direction and position factors. This system's directional and positional reward functions are built based on artificial potential fields (APF) and navigation rules, respectively, which can provide reasoned evaluations for the robot's motion direction and position during training, enabling it to receive comprehensive feedback. Furthermore, the incorporation of Monte-Carlo tree search (MCTS) facilitates the development of a foresighted action strategy, enabling robots to execute actions with long-term planning considerations. Experimental results demonstrate that integrating model inductive bias significantly enhances the navigation performance of MIBE-DRL. Compared to state-of-the-art methods, MIBE-DRL achieves the highest success rate in crowded environments and demonstrates advantages in navigation time and maintaining a safe social distance from humans.