Multi-Camera-Based Human Activity Recognition for Human–Robot Collaboration in Construction
Author:
Jang Youjin1ORCID, Jeong Inbae2, Younesi Heravi Moein1, Sarkar Sajib1, Shin Hyunkyu3, Ahn Yonghan4
Affiliation:
1. Department of Civil, Construction and Environmental Engineering, North Dakota State University, Fargo, ND 58108, USA 2. Department of Mechanical Engineering, North Dakota State University, Fargo, ND 58108, USA 3. Sustainable Smart City Convergence Educational Research Center, Hanyang University ERICA, Ansan 15588, Republic of Korea 4. Department of Architectural Engineering, Hanyang University ERICA, Ansan 15588, Republic of Korea
Abstract
As the use of construction robots continues to increase, ensuring safety and productivity while working alongside human workers becomes crucial. To prevent collisions, robots must recognize human behavior in close proximity. However, single, or RGB-depth cameras have limitations, such as detection failure, sensor malfunction, occlusions, unconstrained lighting, and motion blur. Therefore, this study proposes a multiple-camera approach for human activity recognition during human–robot collaborative activities in construction. The proposed approach employs a particle filter, to estimate the 3D human pose by fusing 2D joint locations extracted from multiple cameras and applies long short-term memory network (LSTM) to recognize ten activities associated with human and robot collaboration tasks in construction. The study compared the performance of human activity recognition models using one, two, three, and four cameras. Results showed that using multiple cameras enhances recognition performance, providing a more accurate and reliable means of identifying and differentiating between various activities. The results of this study are expected to contribute to the advancement of human activity recognition and utilization in human–robot collaboration in construction.
Funder
Korea Institute of Energy Technology Evaluation and Planning
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference57 articles.
1. Maeda, J., Takada, H., Abe, Y., Maeda, J., Takada, H., and Abe, Y. (2004, January 21–25). Applicable Possibility Studies on a Humanoid Robot to Cooperative Work on Construction Site with a Human Worker. Proceedings of the 21st International Symposium on Automation and Robotics in Construction, Jeju Island, Republic of Korea. 2. Bock, T., Bulgalow, A., and Ashida, S. (2017, January 23–25). Façade Cleaning Robot for the Skyscraper. Proceedings of the 19th International Symposium on Automation and Robotics in Construction (ISARC), Washington, DC, USA. 3. Smart Component-Oriented Method of Construction Robot Coordination for Prefabricated Housing;Zhu;Autom. Constr.,2021 4. Human–Robot Collaboration for on-Site Construction;Zhang;Autom. Constr.,2023 5. Brosque, C., Galbally, E., Khatib, O., and Fischer, M. (2020, January 26–28). Human-Robot Collaboration in Construction: Opportunities and Challenges. Proceedings of the HORA 2020—2nd International Congress on Human-Computer Interaction, Optimization and Robotic Applications, Ankara, Turkey.
Cited by
3 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
|
|