Real-time Apparent Resolution Enhancement for Head-mounted Displays

Abstract

The insufficient pixel density of current head-mounted displays is one of the major obstacles in achieving immersive and fully engaging experiences. It is possible to overcome this physical limitation for moving content using software techniques. To this end, previous techniques utilized high-framerate displays and optimized for low-resolution images that, when shown on a display, can significantly increase the apparent spatial resolution. However, so far, all the proposed techniques require expensive optimization, which makes the techniques unsuitable for real-time applications. To overcome this problem, we present a novel method that can improve apparent resolution of such displays in real time. We replace expensive optimizations with a two-step filtering approach. Due to the efficiency of our technique, we can account not only for the motion in the scene but also for any motion in the perceived image introduced by movement of a user. This greatly extends the range of situations where the resolution enhancement can be achieved. In this paper, we present the derivation of the motion-flow-dependent filters and how they can be applied to increase the perceived resolution. To evaluate the performance of our technique, we conducted a user experiment which compares our method to alternative solutions regarding perceived resolution as well as overall quality and demonstrates the advantages of our technique.