Comparative Evaluation of User Perceived Quality Assessment of Design Strategies for HTTP-based Adaptive Streaming

Abstract

HTTP-based Adaptive Streaming (HAS) is the dominant Internet video streaming application. One specific HAS approach, Dynamic Adaptive Streaming over HTTP (DASH), is of particular interest, as it is a widely deployed, standardized implementation. Prior academic research has focused on networking and protocol issues, and has contributed an accepted understanding of the performance and possible performance issues in large deployment scenarios. Our work extends the current understanding of HAS by focusing directly on the impacts of choice of the video quality adaptation algorithm on end-user perceived quality. In congested network scenarios, the details of the adaptation algorithm determine the amount of bandwidth consumed by the application as well as the quality of the rendered video stream. HAS will lead to user-perceived changes in video quality due to intentional changes in quality video segments, or unintentional perceived quality impairments caused by video decoder artifacts such as pixelation, stutters, or short or long stalls in the rendered video when the playback buffer becomes empty. The HAS adaptation algorithm attempts to find the optimal solution to mitigate the conflict between avoiding buffer stalls and maximizing video quality. In this article, we present results from a user study that was designed to provide insights into “best practice guidelines” for a HAS adaptation algorithm. Our findings suggest that a buffer-based strategy might provide a better experience under higher network impairment conditions. For the two network scenarios considered, the buffer-based strategy is effective in avoiding stalls but does so at the cost of reduced video quality. However, the buffer-based strategy does yield a lower number of quality switches as a result of infrequent bitrate adaptations. Participants in buffer-based strategy do notice the drop in video quality causing a decrease in perceived QoE, but the perceived levels of video quality, viewer frustration, and opinions of video clarity and distortion are significantly worse due to artifacts such as stalls in capacity-based strategy. The capacity-based strategy tries to provide the highest video quality possible but produces many more artifacts during playback. The results suggest that player video quality has more of an impact on perceived quality when stalls are infrequent. The study methodology also contributes a unique method for gathering continuous quantitative subjective measure of user perceived quality using a Wii remote.