VR 360º subtitles

Abstract

Subtitle production is an increasingly creative accessibility service. New technologies allow for placing subtitles at any location of the screen in a variety of formats, shapes, typography, font size, and colour. The screen now affords accessible creativity, with subtitles able to provide novel experiences beyond those offered by traditional language translation. Immersive environments multiply 2D subtitles features to new creative viewing modalities. Testing subtitles in eXtended Reality (XR) has pushed existing methods to address user need and enjoyment of audiovisual content in 360º viewing displays. After an overview of existing subtitle features in XR, the article describes the challenges of generating subtitle stimuli to test meaningful user viewing behaviours, based on eye-tracking technology. The approach for the first experimental setup for implementing creative subtitles in XR using eye-tracking is given, in line with novel research questions. The choices made regarding sound, duration and storyboard are described. Conclusions show that testing subtitles in immersive media environments is both a linguistic and an artistic endeavour, which requires an agile framework fostering contrast and comparison of different functionalities. Results of the present, preliminary study shed light on future experimental setups with eye-tracking. Lay summary Subtitling is an increasingly creative accessibility service. Subtitlers use new technologies to place subtitles at any location on the screen. They can also change shape, typography, font size, and colour of the subtitles. This personalisation opens the door for creative solutions and allows novel experiences beyond traditional 2D subtitles. An immersive environment is an illusionary experience that surrounds you and transports you to another place. The challenges when testing subtitles in these new environments are different from those when testing in 2D media, as the user is no longer a passive spectator but an active part of the story. In this article, we give an overview of existing subtitle features in immersive environments. Then we describe the challenges of generating stimuli suitable for testing subtitles in immersive environments using eye tracking technology. We explain the experimental setup used during our experiment as well as the research questions. We describe the choices we made to design our stimuli, such as the sound, the duration, and the storyboard. We conclude that testing subtitles in immersive media environments present linguistics and artistic challenges and that we need to use a framework that allows for rapid contrast of different solutions. Results of the present, preliminary study clear up how to use eye-tracking technology to test subtitles in immersive environments.