Annotating traditional Chinese paintings for immersive virtual exhibition

Abstract

We propose a new method of annotating a masterpiece of traditional Chinese painting with voice dubbings and environmental sounds. The painting was created with moving focus drawing technique without rigorous perspective. A novel algorithm is proposed to infer the 3D space of the painting according to its layout and embed the audio annotations. For exhibition, the masterpiece is scanned into a high-resolution gigapixel image for presenting the drawing details, and we develop an interactive multimedia system with a panning and zooming interface to enable smooth navigation on the giant painting and exploring the historical culture. During the navigation, the system estimates the 3D position of the user's viewpoint from his/her actions, and subsequently synthesizes a realistic stereo audio field according to the viewer's orientation and distance from the annotations in the 3D space. The proposed system provides an immersive user experience by rendering a visual-audio consistent perception.