Automated Translation Accurately Translates Recorded Pediatric Neurosurgery Clinic Conversations between Spanish and English

Abstract

Abstract Objective The purpose of this study is to analyze an automated voice to text translation device by reportingt he translation accuracy for recorded pediatric neurosurgery clinic conversations, classifying errors in translation according to their impact on overall understanding, and comparing the incidence of these errors in English to Spanish vs. Spanish to English conversations. Methods English and Spanish speaking patients at a single academic health system’s outpatient pediatric neurosurgery clinic had their conversations recorded. These recordings were played back to a Google Pixel handheld smartphone with Live Translate voice to text translation software. A certified medical interpreter evaluated recordings for incidence of minor errors, errors impacting understanding, and catastrophic errors affecting patient-provider relationship or care. Two proportion t-testing was used to compare these outcomes. Results 50 patient visits were recorded: 40 English recordings translated to Spanish and 10 Spanish recordings translated to English. The mean transcript length was 4244 ± 992 words. The overall accuracy was 98.2% ± 0.5%. On average, 46 words were missed in translation (1.09% error rate), 31 understanding-altering translation errors (0.73% error rate), and 0 catastrophic errors were made. There was no significant difference in English to Spanish or vice versa. Conclusion Voice to text translation devices accurately translate recorded clinic conversations between Spanish and English and vice versa with high accuracy and very low incidence of errors impacting medical care or understanding. Further study should investigate additional languages as well as compare these devices directly to medical interpreters in live clinic settings.