A Smart Cane Based on 2D LiDAR and RGB-D Camera Sensor-Realizing Navigation and Obstacle Recognition
Author:
Mai Chunming1ORCID, Chen Huaze2, Zeng Lina134, Li Zaijin134, Liu Guojun134, Qiao Zhongliang134ORCID, Qu Yi134, Li Lianhe4, Li Lin134ORCID
Affiliation:
1. College of Physics and Eletronic Engineering, Hainan Normal University, Haikou 571158, China 2. College of Information Science and Technology, Hainan Normal University, Haikou 571158, China 3. Key Laboratory of Laser Technology and Optoelectronic Functional Materials of Hainan Province, Hainan Normal University, Haikou 571158, China 4. Hainan International Joint Research Center for Semiconductor Lasers, Hainan Normal University, Haikou 571158, China
Abstract
In this paper, an intelligent blind guide system based on 2D LiDAR and RGB-D camera sensing is proposed, and the system is mounted on a smart cane. The intelligent guide system relies on 2D LiDAR, an RGB-D camera, IMU, GPS, Jetson nano B01, STM32, and other hardware. The main advantage of the intelligent guide system proposed by us is that the distance between the smart cane and obstacles can be measured by 2D LiDAR based on the cartographer algorithm, thus achieving simultaneous localization and mapping (SLAM). At the same time, through the improved YOLOv5 algorithm, pedestrians, vehicles, pedestrian crosswalks, traffic lights, warning posts, stone piers, tactile paving, and other objects in front of the visually impaired can be quickly and effectively identified. Laser SLAM and improved YOLOv5 obstacle identification tests were carried out inside a teaching building on the campus of Hainan Normal University and on a pedestrian crossing on Longkun South Road in Haikou City, Hainan Province. The results show that the intelligent guide system developed by us can drive the omnidirectional wheels at the bottom of the smart cane and provide the smart cane with a self-leading blind guide function, like a “guide dog”, which can effectively guide the visually impaired to avoid obstacles and reach their predetermined destination, and can quickly and effectively identify the obstacles on the way out. The mapping and positioning accuracy of the system’s laser SLAM is 1 m ± 7 cm, and the laser SLAM speed of this system is 25~31 FPS, which can realize the short-distance obstacle avoidance and navigation function both in indoor and outdoor environments. The improved YOLOv5 helps to identify 86 types of objects. The recognition rates for pedestrian crosswalks and for vehicles are 84.6% and 71.8%, respectively; the overall recognition rate for 86 types of objects is 61.2%, and the obstacle recognition rate of the intelligent guide system is 25–26 FPS.
Funder
Hainan Provincial Natural Science Foundation of China Finance science and technology project of Hainan province National Natural Science Foundation of China specific research fund of The Innovation Platform for Academicians of Hainan Province
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference58 articles.
1. World blindness and visual impairment: Despite many successes, the problem is growing;Ackland;Community Eye Health,2017 2. World Health Organization (2022, November 24). World Report on Vision. Available online: https://www.who.int/zh/news-room/detail/08-10-2019-who-launches-first-world-report-on-vision. 3. Trends in prevalence of blindness and distance and near vision impairment over 30 years: An analysis for the Global Burden of Disease Study;Bourne;Lancet Glob. Health,2021 4. Mai, C., Xie, D., Zeng, L., Li, Z., Li, Z., Qiao, Z., Qu, Y., Liu, G., and Li, L. (2013). Laser Sensing and Vision Sensing Smart Blind Cane: A Review. Sensors, 23. 5. Review on LiDAR-Based Navigation Systems for the Visually Impaired;Jain;SN Comput. Sci.,2023
|
|