D*-KDDPG: An Improved DDPG Path-Planning Algorithm Integrating Kinematic Analysis and the D* Algorithm
-
Published:2024-08-27
Issue:17
Volume:14
Page:7555
-
ISSN:2076-3417
-
Container-title:Applied Sciences
-
language:en
-
Short-container-title:Applied Sciences
Author:
Liu Chunyang12ORCID, Liu Weitao1, Zhang Dingfa1, Sui Xin13ORCID, Huang Yan14, Ma Xiqiang12ORCID, Yang Xiaokang14ORCID, Wang Xiao13
Affiliation:
1. School of Mechatronics Engineering, Henan University of Science and Technology, Luoyang 471003, China 2. Longmen Laboratory, Luoyang 471000, China 3. Key Laboratory of Mechanical Design and Transmission System of Henan Province, Luoyang 471000, China 4. Collaborative Innovation Center of Machinery Equipment Advanced Manufacturing of Henan Province, Luoyang 471000, China
Abstract
To address the limitations of the Deep Deterministic Policy Gradient (DDPG) in robot path planning, we propose an improved DDPG method that integrates kinematic analysis and D* algorithm, termed D*-KDDPG. Firstly, the current work promotes the reward function of DDPG to account for the robot’s kinematic characteristics and environment perception ability. Secondly, informed by the global path information provided by the D* algorithm, DDPG successfully avoids getting trapped in local optima within complex environments. Finally, a comprehensive set of simulation experiments is carried out to investigate the effectiveness of D*-KDDPG within various environments. Simulation results indicate that D*-KDDPG completes strategy learning within only 26.7% of the training steps required by the original DDPG, retrieving enhanced navigation performance and promoting safety. D*-KDDPG outperforms D*-DWA with better obstacle avoidance performance in dynamic environments. Despite a 1.8% longer path, D*-KDDPG reduces navigation time by 16.2%, increases safety distance by 72.1%, and produces smoother paths.
Funder
National Science Foundation of China Technology Projects of Longmen Laboratory
Reference18 articles.
1. Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles;Aradi;IEEE Trans. Intell. Transport. Syst.,2022 2. Immune Deep Reinforcement Learning-Based Path Planning for Mobile Robot in Unknown Environment;Yan;Appl. Soft Comput.,2023 3. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous Control with Deep Reinforcement Learning. arXiv. 4. Zhang, H., Lin, W., and Chen, A. (2018). Path Planning for the Mobile Robot: A Review. Symmetry, 10. 5. Wang, X., Zhang, H., Liu, S., Wang, J., Wang, Y., and Shangguan, D. (2022). Path Planning of Scenic Spots Based on Improved A* Algorithm. Sci. Rep., 12.
|
|