MVS‐SLAM: Enhanced multiview geometry for improved semantic RGBD SLAM in dynamic environment

Abstract

AbstractSimultaneous Localization and Mapping (SLAM) is a crucial technology for intelligent mobile robots to operate successfully in unknown environments. While many excellent SLAM systems have been developed in recent years, most assume that the environment is static, resulting in poor performance in dynamic environments. To address this limitation, we propose multiview stereo (MVS)‐SLAM (MVS‐SLAM), a real‐time semantic RGBD SLAM system with improved‐multiview geometry, built on the RGBD mode of ORB‐SLAM3. MVS‐SLAM tightly integrates semantic and geometric information to tackle the challenges posed by dynamic scenes. To meet the real‐time requirements, the semantic module leverages the latest and fastest object detection network, YOLOv7, to provide semantic prior knowledge for the geometric module. We introduce a novel geometric constraint method that capitalizes on depth images and semantic information to recover three‐dimensional (3D) feature points and initial camera pose. We use a 3D coordinate error threshold to identify dynamic points and remove them using the K‐means clustering algorithm. This approach effectively reduces the impact of dynamic points. We validate MVS‐SLAM using challenging dynamic sequences from the TUM data set, demonstrating that it significantly improves localization accuracy and system robustness in all types of dynamic environments.