Evaluation of Replies to Voice Queries in Gynecologic Oncology by Virtual Assistants Siri, Alexa, Google, and Cortana

Abstract

Women that receive news that they have a malignancy of gynecologic origin can have questions about their diagnosis. These questions might be posed as voice queries to the virtual assistants Siri, Alexa, Google, and Cortana. Because our world has increasingly adopted smart phones and standalone voice query devices, this study focused on the accuracy of audible replies by the virtual assistants (VAs) Siri, Alexa, Google, and Cortana to voice queries related to gynecologic oncology. Twenty-one evaluators analyzed VA audible answers to select voice queries related to gynecologic oncology. Questions were posed in three different ways for each voice query in order to maximize the likelihood of acceptability to the VAs in a 24-question panel. For general queries that were not related to gynecologic oncology, Google provided the most correct audible replies (83.3% correct), followed by Alexa (66.7% correct), Siri (45.8% correct), and Cortana (20.8% correct). For gynecologic oncology-related queries, the accuracy of the VAs was considerably lower: Google provided the most correct audible replies (18.1%), followed by Alexa (6.5%), Siri (5.5%), and Cortana (2.3%). There was a considerable drop in the accuracy of audible replies to oral queries on topics in gynecologic oncology relative to general queries that were not related to gynecologic oncology. There is considerable room for improvement in VA performance, so that caution is advised when using VAs for medical queries in gynecologic oncology. Our specific findings related to gynecologic oncology extend the work of others with regard to the low usability of general medical information obtained from VAs, so that reliance on conversational assistants for actionable medical information represents a safety risk for patients and consumers.