Visual aspect-oriented modeling of explorable extended reality environments

Abstract

AbstractThe availability of various extended reality (XR) systems for tracking users’ and objects’ behavior opens new opportunities for analyzing users’ and objects’ interactions and autonomous actions. Such analysis can be especially useful and attainable to domain experts when it is based on domain knowledge related to a particular application, liberating the analysts from going into technical details of 3D content. Analysis of XR users’ and objects’ behavior can provide knowledge about the users’ experience, interests and preferences, as well as objects’ features, which may be valuable in various domains, e.g., training, design and marketing. However, the available methods and tools for building XR focus on 3D modeling and programming rather than knowledge representation, making them unsuitable for domain-oriented analysis. In this paper, a new visual approach to modeling explorable XR environments is proposed. It is based on a semantic representation of aspects, which extend the primary code of XR environments to register their behavior in a form explorable with reasoning and queries, appropriate for high-level analysis in arbitrary domains. It permits domain experts to comprehend and analyze what happened in an XR environment regarding users’ and objects’ actions and interactions. The approach has been implemented as an extension to MS Visual Studio and demonstrated in an explorable immersive service guide for household appliances. The evaluation results show that the approach enables efficient development of explorable XR and may be useful for people with limited technical skills.