Feedback techniques for continuity and synchronization in multimedia information retrieval

Abstract

Future advances in storage and networking technologies will make it feasible to build multimedia on-demand information servers capable of providing services similar to those of a neighborhood videotape rental store over metropolitan area networks. Such multimedia information servers must not only support retrieval of continuous media units (such as video frames and audio samples), but also preserve synchrony among playback of the different media components constituting a multimedia object. We develop techniques for supporting continuous and synchronous retrieval from multimedia servers. We present feedback techniques by which, during retrieval of multimedia objects from a multimedia server to mediaphones, the multimedia server uses lightweight messages called feedback units transmitted periodically back to it (by mediaphones) to detect impending discontinuities as well as asynchronies at mediaphones. The multimedia server then preventively readjusts media transmission so as to avoid either anomaly, and steers the mediaphones back to synchrony. Given the available buffer sizes at mediaphones and the maximum tolerable asynchrony, we present methods to determine the minimum rate at which feedback units must be transmitted so as to maintain both continuity and synchronization. These feedback techniques remain robust even in the presence of playback rate mismatches and network delay jitter, and their initial simulation for video-audio playback yields a feedback rate of one per 1,000 media units to keep the asynchrony within 250ms, showing that the overhead due to feedback transmission is very small. The constant rate feedback techniques developed in this article form the basis of a prototype on-demand information server being developed at the UCSD Multimedia Laboratory.