A perceptual framework for the auditory display of scientific data

Abstract

A principal requirement of the auditory display of scientific data is an accurate portrayal of the information contained in the data. The characteristics of the display device can significantly affect the faithfulness of the data presentation. Human hearing is a complex nonlinear process and the intuitive comprehension of the display requires consideration of perceptual interactions and the natural connection between the data and the perception of the data. This chapter proposes a perceptual framework for observable and systematic specification and comprehension of sounds in a device-independent auditory display of scientific data. The framework consists of a perceptually scaled sound space and a display model that maps points from the perceptual space to a display device parameter space while preserving the interrelationships between them. The advantages of a perceptual space are described using the example of an established framework for applying perceptually uniform color models to scientific visualization. A perceptual sound space is defined and constructed by scaling a naturally ordered sound model derived from research studies in timbre perception. An auditory display is implemented on a Sun Sparc10 workstation using Csound and samples of musical instruments. The display model consists of an interpolated mapping from perceptual space to display space and a description of the display gamut that is the boundary in perceptual space between points that the display can realize and those that it cannot. The display gamut can be used to analyze the display, compare displays, and optimize data mapping sequences for the display. The concepts are illustrated by a graphic visualization of the gamut of the implemented display as a geometric shape in the perceptual sound space.