RoboTwin Metaverse Platform for Robotic Random Bin Picking
-
Published:2023-07-29
Issue:15
Volume:13
Page:8779
-
ISSN:2076-3417
-
Container-title:Applied Sciences
-
language:en
-
Short-container-title:Applied Sciences
Author:
Tsai Cheng-Han12, Hernandez Eduin E.2, You Xiu-Wen2, Lin Hsin-Yi2, Chang Jen-Yuan13ORCID
Affiliation:
1. Department of Power Mechanical Engineering, National Tsing Hua University, Hsinchu 30013, Taiwan 2. Mechanical and Mechatronics System Research Labs (MMSL), Industrial Technology Research Institute (ITRI), Hsinchu 310401, Taiwan 3. Mechanical & Computer-Aided Engineering, National Formosa University, Yulin 632, Taiwan
Abstract
Although vision-guided robotic picking systems are commonly used in factory environments, achieving rapid changeover for diverse workpiece types can still be challenging because the manual redefinition of vision software and tedious collection and annotation of datasets consistently hinder the automation process. In this paper, we present a novel approach for rapid workpiece changeover in a vision-guided robotic picking system using the proposed RoboTwin and FOVision systems. The RoboTwin system offers a realistic metaverse scene that enables tuning robot movements and gripper reactions. Additionally, it automatically generates annotated virtual images for each workpiece’s pickable point. These images serve as training datasets for an AI model and are deployed to the FOVision system, a platform that includes vision and edge computing capabilities for the robotic manipulator. The system achieves an instance segmentation mean average precision of 70% and a picking success rate of over 80% in real-world detection scenarios. The proposed approach can accelerate dataset generation by 80 times compared with manual annotation, which helps to reduce simulation-to-real gap errors and enables rapid line changeover within flexible manufacturing systems in factories.
Funder
Industrial Technology Research Institute
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference17 articles.
1. Rivera-Calderón, S., Lázaro, R.P.-S., and Vazquez-Hurtado, C. (2022, January 28–31). Online assessment of computer vision and robotics skills based on a digital twin. Proceedings of the 2022 IEEE Global Engineering Education Conference (EDUCON), Tunis, Tunisia. 2. Bansal, R., Khanesar, M.A., and Branson, D. (2019, January 5–7). Ant Colony Optimization Algorithm for Industrial Robot Programming in a Digital Twin. Proceedings of the 2019 25th International Conference on Automation and Computing (ICAC), Lancaster, UK. 3. Borangiu, T., Răileanu, S., Silişteanu, A., Anton, S., and Anton, F. (2020, January 8–10). Smart Manufacturing Control with Cloud-embedded Digital Twins. Proceedings of the 2020 24th International Conference on System Theory, Control and Computing (ICSTCC), Sinaia, Romania. 4. Paradis, S., Hwang, M., Thananjeyan, B., Ichnowski, J., Seita, D., Fer, D., Low, T., Gonzalez, J.E., and Goldberg, K. (June, January 30). Intermittent Visual Servoing: Efficiently Learning Policies Robust to Instrument Changes for High-precision Surgical Manipulation. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China. 5. Lee, C.-T., Tsai, C.-H., and Chang, J.-Y. (2020, January 24–25). A CAD-Free Random Bin Picking System for Fast Changeover on Multiple Objects. Proceedings of the ASME 2020 29th Conference on Information Storage and Processing Systems, Virtual, Online.
|
|