Evaluation of a Language Translation App in an Undergraduate Medical Communication Course: Proof-of-Concept and Usability Study (Preprint)

Abstract

Language barriers in medical encounters pose risks for interactions with patients, their care, and their outcomes. Because human translators, the gold standard for mitigating language barriers, can be cost- and time-intensive, mechanical alternatives such as language translation apps (LTA) have gained in popularity. However, adequate training for physicians in using LTAs remains elusive.

A proof-of-concept pilot study was designed to evaluate the use of a speech-to-speech LTA in a specific simulated physician-patient situation, particularly its perceived usability, helpfulness, and meaningfulness, and to assess the teaching unit overall.

Students engaged in a 90-min simulation with a standardized patient (SP) and the LTA iTranslate Converse. Thereafter, they rated the LTA with six items—<i>helpful</i>, <i>intuitive</i>, <i>informative</i>, <i>accurate</i>, <i>recommendable</i>, and <i>applicable—</i>on a 7-point Likert scale ranging from 1 (<i>don’t agree at all</i>) to 7 (<i>completely agree</i>) and could provide free-text responses for four items: general impression of the LTA, the LTA’s benefits, the LTA’s risks, and suggestions for improvement. Students also assessed the teaching unit on a 6-point scale from 1 (<i>excellent</i>) to 6 (<i>insufficient</i>). Data were evaluated quantitatively with mean (SD) values and qualitatively in thematic content analysis.

Of 111 students in the course, 76 (68.5%) participated (59.2% women, age 20.7 years, SD 3.3 years). Values for the LTA’s being <i>helpful</i> (mean 3.45, SD 1.79), <i>recommendable</i> (mean 3.33, SD 1.65) and <i>applicable</i> (mean 3.57, SD 1.85) were centered around the average of 3.5. The items <i>intuitive</i> (mean 4.57, SD 1.74) and <i>informative</i> (mean 4.53, SD 1.95) were above average. The only below-average item concerned its <i>accuracy</i> (mean 2.38, SD 1.36). Students rated the teaching unit as being excellent (mean 1.2, SD 0.54) but wanted practical training with an SP plus a simulated human translator first. Free-text responses revealed several concerns about translation errors that could jeopardize diagnostic decisions. Students feared that patient-physician communication mediated by the LTA could decrease empathy and raised concerns regarding data protection and technical reliability. Nevertheless, they appreciated the LTA’s cost-effectiveness and usefulness as the best option when the gold standard is unavailable. They also reported wanting more medical-specific vocabulary and images to convey all information necessary for medical communication.

This study revealed the feasibility of using a speech-to-speech LTA in an undergraduate medical course. Although human translators remain the gold standard, LTAs could be valuable alternatives. Students appreciated the simulated teaching and recognized the LTA’s potential benefits and risks for use in real-world clinical settings. To optimize patients’ and health care professionals’ experiences with LTAs, future investigations should examine specific design options for training interventions and consider the legal aspects of human-machine interaction in health care settings.