Enabling the blind to see gestures

Abstract

Human discourse is an embodied activity emerging from the embodied imagery and construction of our talk. Gesture and speech are coexpressive, conveying this imagery and meaning simultaneously. Mathematics instruction and discourse typically involve two modes of communication: speech and graphical presentation. Our goal is to assist Individuals who are Blind or Severely Visually Impaired (IBSVI) to access such instruction/communication. We employ a haptic glove interface to furnish the IBSVI with awareness of the deictic gestures performed by the instructor over the graphic in conjunction with speech. We present a series of studies spanning two years where we show how our Haptic Deictic System (HDS) can support learning in inclusive classrooms where IBSVI receive instruction alongside sighted students. We discuss how the introduction of the HDS was advantageous to all parties: IBSVI, instructor, and sighted students. The HDS created more learning opportunities, increasing mutual understanding and promoting greater engagement.