Effects of Speech- and Text-Based Interaction Modes in Natural Language Human-Computer Dialogue

Abstract

Objective: This study examined the effects of user production (speaking and typing) and user reception (listening and reading) modes on natural language human-computer dialogue. Background: Text-based dialogue is often more efficient than speech-based dialogue, but the latter is more dynamic and more suitable for mobile environments and hands-busy situations. The respective contributions of user production and reception modes have not previously been assessed. Method: Eighteen participants performed several information search tasks using a natural language information system in four experimental conditions: phone (speaking and listening), Web (typing and reading), and mixed (speaking and reading or typing and listening). Results: Mental workload was greater and participants' repetitions of commands were more frequent when speech (speaking or listening) was used for both the user production and reception modes rather than text (typing or reading). Completion times were longer for listening than for reading. Satisfaction was lower, utterances were longer, and the interaction error rate was higher for speaking than typing. Conclusion: The production and reception modes both contribute to dialogue and mental workload. They have distinct contributions to performance, satisfaction, and the form of the discourse. Application: The most efficient configuration for interacting in natural language would appear to be speech for production and system prompts in text, as this combination decreases the time on task while improving dialogue involvement.