Comparison of the Impact of Wikipedia, UpToDate, and a Digital Textbook on Short-Term Knowledge Acquisition Among Medical Students: Randomized Controlled Trial of Three Web-Based Resources (Preprint)

Abstract

Web-based resources are commonly used by medical students to supplement curricular material. Three commonly used resources are UpToDate (Wolters Kluwer Inc), digital textbooks, and Wikipedia; there are concerns, however, regarding Wikipedia’s reliability and accuracy.

The aim of this study was to evaluate the impact of Wikipedia use on medical students’ short-term knowledge acquisition compared with UpToDate and a digital textbook.

This was a prospective, nonblinded, three-arm randomized trial. The study was conducted from April 2014 to December 2016. Preclerkship medical students were recruited from four Canadian medical schools. Convenience sampling was used to recruit participants through word of mouth, social media, and email. Participants must have been enrolled in their first or second year of medical school at a Canadian medical school. After recruitment, participants were randomized to one of the three Web-based resources: Wikipedia, UpToDate, or a digital textbook. During testing, participants first completed a multiple-choice questionnaire (MCQ) of 25 questions emulating a Canadian medical licensing examination. During the MCQ, participants took notes on topics to research. Then, participants researched topics and took written notes using their assigned resource. They completed the same MCQ again while referencing their notes. Participants also rated the importance and availability of five factors pertinent to Web-based resources. The primary outcome measure was knowledge acquisition as measured by posttest scores. The secondary outcome measures were participants’ perceptions of importance and availability of each resource factor.

A total of 116 medical students were recruited. Analysis of variance of the MCQ scores demonstrated a significant interaction between time and group effects (P<.001, ηg2=0.03), with the Wikipedia group scoring higher on the MCQ posttest compared with the textbook group (P<.001, d=0.86). Access to hyperlinks, search functions, and open-source editing were rated significantly higher by the Wikipedia group compared with the textbook group (P<.001). Additionally, the Wikipedia group rated open access editing significantly higher than the UpToDate group; expert editing and references were rated significantly higher by the UpToDate group compared with the Wikipedia group (P<.001).

Medical students who used Wikipedia had superior short-term knowledge acquisition compared with those who used a digital textbook. Additionally, the Wikipedia group trended toward better posttest performance compared with the UpToDate group, though this difference was not significant. There were no significant differences between the UpToDate group and the digital textbook group. This study challenges the view that Wikipedia should be discouraged among medical students, instead suggesting a potential role in medical education.