Metaverse Interface with Haptic and Rigid Sense Feedback at a Low Cost

Abstract

In the metaverse era, anyone should be able to easily access the metaverse for non-discrimination. This study proposes a prototype of a metaverse interface that can be implemented at a low price. It shows reasonable performance. It includes the following 4 modules. 1. Haptic feedback: Piezo sensors have been used for tactile information of various textures, which is transported as an audio data format, and reconstructed at the fingertips by an actuator (LRA motor) attached to a glove. 2. Force feedback: The module was designed to control the servomotor through 3D modeling. It provides force feedback by controlling the servo motor to grab the hand as much as the thickness of the object on Unity, the game engine. 3. Controller: Reproduce finger movement in Unity by measuring resistance change according to finger bending using a Flex sensor. Implement Yawing/Pitching/Rolling movement in Unity through MPU-6050(IMU) sensor. 4. Pose estimation: The user's current movement captured by the webcam is used for pose estimation through the Mediapipe library. It is integrated with the hand posture received serially from the controller to render the user posture inside the Unity game. The developed interface can be applied to life logging service and mirror world service, which are one of the most important applications in the metaverse.