Affiliation:
1. Faculty of Electrical Engineering and Information Technology, Slovak University of Technology in Bratislava, 841 04 Bratislava, Slovakia
Abstract
This article presents the design and implementation of an innovative human–machine interface (HMI) in mixed reality for a robot model operating within Robot Operating System 2 (ROS 2). The interface is specifically developed for compatibility with Microsoft HoloLens 2 hardware and leverages the Unity game engine alongside the Mixed Reality Toolkit (MRTK) to create an immersive mixed reality application. The project uses the Turtlebot 3 Burger model robot, simulated within the Gazebo virtual environment, as a representative mechatronic system for demonstration purposes. Communication between the mixed reality application and ROS 2 is facilitated through a publish–subscribe mechanism, utilizing ROS TCP Connector for message serialization between nodes. This interface not only enhances the user experience by allowing for the real-time monitoring and control of the robotic system but also aligns with the principles of Industry 5.0, emphasizing human-centric and inclusive technological advancements. The practical outcomes of this research include a fully functional mixed reality application that integrates seamlessly with ROS 2, showcasing the potential of mixed reality technologies in advancing the field of industrial automation and human–machine interaction.
Funder
Scientific Grant Agency of the Ministry of Education, Research, Development and Youth of the Slovak Republic
Reference44 articles.
1. Extended reality applications in industry 4.0.–A systematic literature review;Reta;Telemat. Inform.,2022
2. (2023, May 08). Revolutionize Manufacturing with Augmented Reality Solutions. Available online: https://www.rockwellautomation.com/en-us/products/software/factorytalk/innovationsuite/augmented-reality.html.
3. Stark, E., Kučera, E., Haffner, O., Drahoš, P., and Leskovský, R. (2020). Using Augmented Reality and Internet of Things for Control and Monitoring of Mechatronic Devices. Electronics, 9.
4. (2020, June 02). The Foundry. VR? AR? MR? Sorry, I’m Confused. Available online: https://www.foundry.com/insights/vr-ar-mr/vr-mr-ar-confused.
5. Gilmore, J.N., and Blair, B.W. (2024). Make Room for VR: Constructing Domestic Space and Accessibility in Virtual Reality Headset Tutorials. Telev. New Media, 15274764241232333.