Improving Live Augmented Reality With Neural Configuration Adaptation

Abstract

Instead of relying on remote clouds, today's augmented reality (AR) applications send videos to nearby edge servers for analysis to optimize user's quality of experience (QoE). Lots of studies have been conducted to help adaptively choose the best video configuration, e.g., resolution and frame per second (fps). However, prior works only consider network bandwidth and ignores the video content itself. In this chapter, the authors design Cuttlefish, a system that generates video configuration decisions using reinforcement learning (RL) based on network condition as well as the video content. Cuttlefish does not rely on any pre-programmed models or specific assumptions on the environments. Instead, it learns to make configuration decisions solely through observations of the resulting performance of historical decisions. Cuttlefish automatically learns the adaptive configuration policy for diverse AR video streams and obtains a gratifying QoE. The experimental results show that Cuttlefish achieves a 18.4%-25.8% higher QoE than the other prior designs.