On the Consistency of 360 Video Quality Assessment in Repeated Subjective Tests: A Pilot Study
-
Published:2024-01-08
Issue:1
Volume:11
Page:
-
ISSN:2410-0218
-
Container-title:EAI Endorsed Transactions on Industrial Networks and Intelligent Systems
-
language:
-
Short-container-title:EAI Endorsed Trans Ind Net Intel Syst
Author:
Elwardy Majed,Zepernick Hans-Juergen,Chu Thi My Chinh,Hu Yan
Abstract
Immersive media such as virtual reality, augmented reality, and 360◦ video have seen tremendous technological developments in recent years. Furthermore, the advances in head-mounted displays (HMDs) offer the users increased immersive experiences compared to conventional displays. To develop novel immersive media systems and services that satisfy the expectations of the users, it is essential to conduct subjective tests revealing users’ perceived quality of immersive media. However, due to the new viewing dimensions provided by HMDs and the potential of interacting with the content, a wide range of subjective tests are required to understand the many aspects of user behavior in and quality perception of immersive media. The ground truth obtained by such subjective tests enable the development of optimized immersive media systems that fulfill the expectations of the users. This article focuses on the consistency of 360◦ video quality assessment to reveal whether users’ subjective quality assessment of such immersive visual stimuli changes fundamentally over time or is kept consistent with each user having their own behavior signature. A pilot study was conducted under pandemic conditions with participants given the task of rating the quality of 360◦ video stimuli on an HMD in standing and seated viewing. The choice of conducting a pilot study is motivated by the fact that immersive media impose high cognitive load on the participants and the need to keep the number of participants under pandemic conditions as low as possible. To gain insight into the consistency of the participants’ 360◦ video assessment over time, three sessions were held for each participant and each viewing condition with long and short breaks between sessions. In particular, the opinion scores and head movements were recorded for each participant and each session in standing and seated viewing. The statistical analysis of this data leads to the conjecture that the quality rating stays consistent throughout these sessions with each participant having their own quality assessment signature. The head movements, indicating the participants’ scene exploration during the quality assessment task, also remain consistent for each participant according their individual narrower or wider scene exploration signature. These findings are more pronounced for standing viewing than for seated viewing. This work supports the role of pilot studies being a useful approach of conducting pre-tests on immersive media quality under opportunity-limited conditions and for the planning of subsequent full subjective tests with a large panel of participants. The annotated RQA360 dataset containing the data recorded in the repeated subjective tests is made publicly available to the research community.
Publisher
European Alliance for Innovation n.o.
Subject
Computer Networks and Communications,Computer Science Applications,Information Systems,Control and Systems Engineering
Reference38 articles.
1. Aloqaily, M., Bouachir, O., Karray, F., Al Ridhawi, I. and Saddik, A.E. (2023) Integrating Digital Twin and Advanced Intelligent Technologies to Realize the Metaverse. IEEE Consumer Electronics Magazine 12(6): 47–55. 2. Han, Y., Niyato, D., Leung, C., Miao, C. and Kim, D.I. (2022) A Dynamic Resource Allocation Framework for Synchronizing Metaverse with IoT Service and Data. In Proc. IEEE Int. Conf. on Communications (Seoul, Republic of Korea): 298–301. 3. 3GPP TR 26.918 V16.0.0 (2018) Virtual Reality (VR) Media Services Over 3GPP (Release 16), 3rd Generation Partnership Project, Technical Specification Group Services and System Aspects. 4. Yrjölä, S., Ahokangas, P. and Matinmikko-Blue, M. (Jun. 2020) White Paper on Business of 6G. (6G Research Visions, No. 3), University of Oulu, Finland. 5. Duanmu, Z., Zeng, K., Ma, K., Rehman, A. and Wang, Z. (2017) A Quality-of-Experience Index for Streaming Video. IEEE Journal of Selected Topics in Signal Processing 11(1): 154–166.
|
|