XRSISE: An XR Training System for Interactive Simulation and Ergonomics Assessment

Abstract

The use of virtual reality (VR) techniques for industrial training provides a safe and cost effective solution that contributes to increased engagement and knowledge retention levels. However, the process of experiential learning in a virtual world without biophysical constraints might contribute to muscle strain and discomfort, if ergonomic risk factors are not considered in advance. Under this scope, we have developed a digital platform which employs extended reality (XR) technologies for the creation and delivery of industrial training programs, by taking into account the users and workplace specificities through the adaptation of the 3D virtual world to the real environment. Our conceptual framework is composed of several inter-related modules: 1) the XR tutorial creation module, for automatic recognition of the sequence of actions composing a complex scenario while this is demonstrated by the educator in VR, 2) the XR tutorial execution module, for the delivery of visually guided and personalized XR training experiences, 3) the digital human model (DHM) based simulation module for creation and demonstration of job task simulations avoiding the need of an actual user and 4) the biophysics assessment module for ergonomics analysis given the input received from the other modules. Three-dimensional reconstruction and aligned projection of the objects situated in the real scene facilitated the imposition of inherent physical constraints, thereby allowed to seamlessly blend the virtual with the real world without losing the sense of presence.