Comparing UpToDate and DynaMed Plus as medical knowledge resources for clinical decision support: cross-over randomized trial (Preprint)

Abstract

Electronic medical knowledge resources are frequently used for clinical decision support. Costs vary substantially among vendors, warranting periodic assessment of institution-wide adoption.

To compare two medical knowledge resources, UpToDate and DynaMed Plus, with respect to facilitating accuracy and efficiency of answering standardized clinical questions and user experience.

Physicians in training at a single academic medical center were randomized to first use one of the two medical knowledge resources to answer six standardized clinical case questions. They were surveyed about their experience using the resource, then completed the questions using the other resource, followed by the user experience survey again. The percentage of accurate answers and time required to answer each question were recorded. The surveys assessed ease of use, enjoyment using the resource, quality of information, and ability to assess the level of evidence. Given the cross-over design, tests of carry-over effects were performed. For open-ended survey items regarding overall user experience, themes were identified, and sentiment analyses were performed.

Twenty-six physicians in training participated, with a mean (standard deviation, SD) of 5.8 (2.5) years of prior experience using UpToDate. Accuracy of answers when using the two resources generally differed by 4 percentage points or less. For all but one question, there were no significant differences in the time required for completion. Most participants perceived both resources to be easy to use, have high quality of information, and felt able to assess to the level of evidence contained in the information. A greater proportion of participants (23/26, 88%) endorsed enjoyment of use when using UpToDate compared to when using DynaMed Plus (16/26, 62%). Participants were significantly less likely to enjoy DynaMed Plus if they were in the randomization group assigned to use UpToDate first (p=0.006). Themes emerging from open-ended survey comments included interface/information presentation, coverage of clinical topics, search functions, and utility for clinical decision-making. Overall, the majority (59%) of open-ended comments expressed an overall preference for UpToDate, while only 19% expressed an overall preference for DynaMed Plus.

DynaMed Plus is non-inferior to UpToDate with respect to ability to achieve accurate answers, time required for answering clinical questions, ease of use, quality of information, and ability to asses level of evidence. However, user experience was more positive with UpToDate, leading to a majority of users stating a preference for UpToDate. Future studies of electronic medical knowledge resources should continue to emphasize evaluation of usability and user experience.