Computational Requirements and Synchronization Issues for Virtual Acoustic Displays

Abstract

This paper addresses two main issues concerning virtual acoustic displays. First, we discuss the computational requirements including sound generation (or synthesis), environmental effects modeling, and three-dimensional (3-D) sound localization. The computational analysis reveals that acoustic processing delays of at least 66 ms are expected with today's technology. This analysis motivates the second issue: how much computational time is available for executing the acoustic process, assuming the requirement for perceptually perfect audiovisual synchronization? A psychoacoustic experiment designed to quantify the tolerable audiovisual delay indicates that an acoustic impact event must occur within an average of 175 ms of the visual event in order for the events to be perceived as synchronous. The most highly trained observers detect desynchrony with an audiovisual delay as low as 100 ms. The results of the computational requirement analysis and the psychoacoustic synchronization experiment provide important information for designers and researchers of virtual acoustic displays.