A Deep Reinforcement Learning Approach to Optimal Morphologies Generation in Reconfigurable Tiling Robots

Abstract

Reconfigurable robots have the potential to perform complex tasks by adapting their morphology to different environments. However, designing optimal morphologies for these robots is challenging due to the large design space and the complex interactions between the robot and the environment. An in-house robot named Smorphi, having four holonomic mobile units connected with three hinge joints, is designed to maximize area coverage with its shape-changing features using transformation design principles (TDP). The reinforcement learning (RL) approach is used to identify the optimal morphologies out of a vast combination of hinge angles for a given task by maximizing a reward signal that reflects the robot’s performance. The proposed approach involves three steps: (i) Modeling the Smorphi design space with a Markov decision process (MDP) for sequential decision-making; (ii) a footprint-based complete coverage path planner to compute coverage and path length metrics for various Smorphi morphologies; and (iii) pptimizing policies through proximal policy optimization (PPO) and asynchronous advantage actor–critic (A3C) reinforcement learning techniques, resulting in the generation of energy-efficient, optimal Smorphi robot configurations by maximizing rewards. The proposed approach is applied and validated using two different environment maps, and the results are also compared with the suboptimal random shapes along with the Pareto front solutions using NSGA-II. The study contributes to the field of reconfigurable robots by providing a systematic approach for generating optimal morphologies that can improve the performance of reconfigurable robots in a variety of tasks.