Depth perception optimization of mixed reality simulation systems based on multiple‐cue fusion

Abstract

AbstractMixed reality technology can be applied to simulation training, improve surgical performance, enhance 3D game experience, and so on, attracting extensive attention of researchers. The perception of the user about head‐mounted display MR using HoloLens is critical, especially for precision applications such as virtual hoisting. Designing and adding appropriate depth cues in MR scenes is an effective way to improve users' depth perception. In this study, taking virtual hoisting training system as an example, the depth perception strategy of multi‐cue fusion is proposed to improve the perception effect. Based on the mechanism of human depth perception, five kinds of depth cues are designed. The depth perception effect of adding single cue is studied by perceptual matching experiment. Based on the principle of fuzzy clustering, a multiple‐cue comprehensive depth optimization strategy on viewing distance scale is proposed. Finally, the perceptual matching results demonstrate the effectiveness of the multi‐cue fusion strategy, and the average error is reduced by 20.68% compared with the single‐cue strategy, which can significantly improve the spatial depth perception. This research can provide a reference for improving users' depth perception in interactive MR simulation systems.