Multimedia Human-Computer Interaction for Presence and Exploration in a Telemuseum

Abstract

It is not always feasible or practical to be present at a location of interest. Telepresence makes such possible. This paper presents the design and implementation of a telepresence system in which a mobile manipulator robot is utilized to explore a museum at a remote site, sending live video and audio on demand to a user at the local site. The user's sense of presence is also enhanced through the capability to manipulate objects remotely via the robot's gripper. The display system is a slightly modified, off-the-shelf mediascope, a television display system that can be worn like a pair of glasses. The head movements of the user are monitored using color tracking, and the displayed images are updated according to the motions of the head. This is achieved by interpreting the human head movements and transforming them into control signals for appropriate movements of the robot and its mounted color camera. Novel features of the presented system include utilization of human implicit commands, availability and cost-effectiveness of the system, feeling of presence, and intelligence fusion of the human user (where to go and what to do) with the robot's intelligence (local obstacle avoidance and navigation). The multimedia interface and control of the system are described in detail. Additionally, a number of research issues that require further investigation in order to realize attainable, high fidelity telepresence systems are discussed in this paper.