Using Mixed Reality for Control and Monitoring of Robot Model Based on Robot Operating System 2

Author:

Janecký Dominik1ORCID,Kučera Erik1ORCID,Haffner Oto1ORCID,Výchlopeňová Erika1,Rosinová Danica1ORCID

Affiliation:

1. Faculty of Electrical Engineering and Information Technology, Slovak University of Technology in Bratislava, 841 04 Bratislava, Slovakia

Abstract

This article presents the design and implementation of an innovative human–machine interface (HMI) in mixed reality for a robot model operating within Robot Operating System 2 (ROS 2). The interface is specifically developed for compatibility with Microsoft HoloLens 2 hardware and leverages the Unity game engine alongside the Mixed Reality Toolkit (MRTK) to create an immersive mixed reality application. The project uses the Turtlebot 3 Burger model robot, simulated within the Gazebo virtual environment, as a representative mechatronic system for demonstration purposes. Communication between the mixed reality application and ROS 2 is facilitated through a publish–subscribe mechanism, utilizing ROS TCP Connector for message serialization between nodes. This interface not only enhances the user experience by allowing for the real-time monitoring and control of the robotic system but also aligns with the principles of Industry 5.0, emphasizing human-centric and inclusive technological advancements. The practical outcomes of this research include a fully functional mixed reality application that integrates seamlessly with ROS 2, showcasing the potential of mixed reality technologies in advancing the field of industrial automation and human–machine interaction.

Funder

Scientific Grant Agency of the Ministry of Education, Research, Development and Youth of the Slovak Republic

Publisher

MDPI AG

Reference44 articles.

1. Extended reality applications in industry 4.0.–A systematic literature review;Reta;Telemat. Inform.,2022

2. (2023, May 08). Revolutionize Manufacturing with Augmented Reality Solutions. Available online: https://www.rockwellautomation.com/en-us/products/software/factorytalk/innovationsuite/augmented-reality.html.

3. Stark, E., Kučera, E., Haffner, O., Drahoš, P., and Leskovský, R. (2020). Using Augmented Reality and Internet of Things for Control and Monitoring of Mechatronic Devices. Electronics, 9.

4. (2020, June 02). The Foundry. VR? AR? MR? Sorry, I’m Confused. Available online: https://www.foundry.com/insights/vr-ar-mr/vr-mr-ar-confused.

5. Gilmore, J.N., and Blair, B.W. (2024). Make Room for VR: Constructing Domestic Space and Accessibility in Virtual Reality Headset Tutorials. Telev. New Media, 15274764241232333.

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3