The Limitations of Representing Sound and Notation on Screen

Abstract

Animated screen-based notation and visual representation of sound provide an important solution to visualising a range of musical phenomena and techniques including continuous parametrical changes, synchronisation with prerecorded audio or live processing, and nonlinear formal organisation. The limitations of human visual capabilities, however, place some constraints upon the efficacy of screen-based representation, particularly in regard to notation reading. Findings from sightreading studies exploring the manner in which notation is encoded, processed and executed are examined with the aim of identifying the perceptual and practical boundaries of presenting animated notation on screen. The development of efficient visual representation is proposed as an important requirement for alleviating the issues created by the time constraints of reading on screen. Studies in semantics and cross-modal activation are discussed as a foundation for the expansion of approaches to the visualisation of sound.