Effects of Sound Quality on the Accuracy of Telephone Captions Produced by Automatic Speech Recognition: A Preliminary Investigation

Abstract

Purpose: Automatic speech recognition (ASR) is commonly used to produce telephone captions to provide telecommunication access for individuals who are d/Deaf and hard of hearing (DHH). However, little is known about the effects of degraded telephone audio on the intelligibility of ASR captioning. This research note investigates the accuracy of telephone captions produced by ASR under degraded audio conditions. Method: Packet loss, delay, and repetition are common sources of degradation in sound quality for telephone audio. Eleven sets of wideband filtered sentences were degraded by high and low levels of simulated packet loss, delay, and repetition. These sets, along with a clean set of sentences, were submitted to ASR, and the accuracy of the resulting output was evaluated using three metrics: a word recognition score, word error rate, and word information loss. Results: The resulting pattern of data indicated the relative impact of each degraded condition on message intelligibility. The high and low packet loss conditions had the largest effect on message intelligibility. This finding was interpreted to indicate that packet loss can have a substantial impact on the accuracy of telephone captions produced with ASR. Conclusions: The results of this investigation point to a potential area of improvement in service quality that could have a substantial impact on telecommunication services for consumers who are DHH. Further research in this area is needed to provide additional information concerning the scope and impact of packet loss on the accuracy of telephone captioning produced by ASR. Supplemental Material: https://doi.org/10.23641/asha.21699557