A synthetic visual environment with hand gesturing and voice input

Abstract

This paper describes a practical synthetic visual environment for use in CAD and teleoperation. Instead of using expensive head mounted display systems, we use a standard display and compute smooth shaded images using an AT&T Pixel Machine. The interface uses a VPL DataGlove [9] to track the hand, bringing the synthetic world into the same space as the hand. Hand gesturing is used to implement a virtual control panel, and some 3D modeling tasks. When simple speech recognition was added it markedly improved the interface. We also outline what extensions might be needed for using this kind of interface for teleoperation.