When and how well do people see the onset of gestures?

Abstract

We studied if and when people detect the beginning of a gesture, in our case a sign in Sign Language of the Netherlands (SLN), by presenting movie fragments consisting of sequences of rest positions, fidgets, and signs to deaf signers, hearing signers and non-signers. Participants were instructed to respond as soon as they saw that a SLN sign had begun. All participants showed themselves highly capable of responding to sign beginnings. Signs that are two-handed, performed in signing space, have a highly marked hand shape, and contain path movement were discriminated best. Considering a sign as having a preparation, a stroke, and a recovery, response times showed strong clusters around 500 milliseconds after the beginning of sign preparation, or 200 ms after the onset of the stroke. The non-signers needed more time before responding; deaf signers took more time than hearing signers. Response time was influenced by three factors (shorter for signs that have a highly marked hand shape, are one-handed, and are preceded by fidgets). The results show that it is possible for people to discriminate fidgeting and signs based on appearance, even if one does not know sign language. No single feature of the movement appears necessary to detect the beginning of a sign. In most cases visual information available up to an early stage of the stroke is sufficient but in some cases the information in the preparation is enough.