MIVAS: A Multi-Modal Immersive Virtual Assembly System

Abstract

Evaluation and planning of assembly processes in virtual environments have become an active research area in engineering community. However, planning of complex assemblies in virtual environments, especially large-scale virtual environments, is still hindered by limitations like unnatural user interaction, insufficient frame rates, and deficiencies in processing of assembly constraints. In this paper, we present MIVAS, a Multi-modal Immersive Virtual Assembly System. By viewing the virtual assembly system as a finite state machine, we incorporate tracked devices, force feedback dataglove, voice commands, human sounds, fully immersive 4-sided CAVE, together with optimization techniques for both complex assembly models and assembly operations to provide for engineers an intuitive and natural way of assembly evaluation and planning. Testing scenarios on disassembling different components of an intelligent hydraulic excavator are described. Special attention is paid upon such technical issues as interface between CAD packages and the CAVE virtual environment, natural and intuitive user interaction including realistic virtual hand interaction and force feedback, intelligent navigation for assembly operations, and real-time display of complex assemblies.