OCCLUSION SCREENING USING 3D CITY MODELS AS A REFERENCE DATABASE FOR MOBILE AR-APPLICATIONS

Abstract

Abstract. Creating an immersive Augmented Reality (AR) experience requires aligning the digital content with the real environment so that the digital content appears and interacts in a similar way to a real object. For that, the virtual object must persist in its position across sessions and be occluded completely or partly, depending on whether a real-world object is in the line of sight. This research makes use of the cutting-edge technology available in the field of AR/MR, the Microsoft HoloLens 2.0, where the in-device Time of Flight (ToF) camera is used to scan the environment to create a Spatial Map. The CityGML LOD1 model enriches the Spatial Map and, in addition, is used as an occlusion mask via a custom rendering pipeline. Spatial Anchors can be tracked using the Simultaneous Localization And Mapping (SLAM) technique, used in conjunction with World Locking Tools to anchor the whole scene to the real-world coordinates. The developed occlusion effect is used in urban planning scenarios to introduce a new area design featuring a car-free environment. The proposed method is evaluated based on performance indicators, including frames per second (FPS) since it is highly correlated to user comfort. The findings of the users’ study demonstrate that the occlusion effect achieves its purpose since most of the participants reported enhancements in the depth perception and overall experience by enabling the occlusion screening.