Comparison of Online Tests of Very Short Answer versus Single Best Answers for Medical Students in a Pharmacology Course over One Year

Abstract

Single best answers (single choice, SC) are the classical tools used in medical examinations on all levels of education. In contrast, very short answer (VSA) probably requires sound prior knowledge and deeper learning than SC, and VSA should make cueing and guessing impossible. Hence, in a basic pharmacology course, we wanted to compare the SC and VSA formats directly at the end of the course and one year later. Medical students (n = 211) were given a formative online test. Two groups were randomly formed (A and B). Participants in group A were first given fifteen single choices (one out of five) pharmacology questions and thereafter fifteen very short answer questions (open question which were to be answered online and semiautomatically assessed). Very similar questions with regard to learning objectives but in opposite order were given to group B. After one year, about half of students from group A were again given the very same questions (AA) or the opposite questions (AB). Likewise, group B was again tested with the opposite questions (BA) or the same (BB). The SC questions in groups A, AA, AB, B, BA, and BB were in sum easier to answer than the corresponding VSA questions. Repeating the test after one year with the same students increased retention of right answers by about 1.5 points. In summary, direct comparison questions in the VSA format are more difficult for our students to answer than questions in the SC format, conceivably because cueing and guessing are eliminated. Knowledge retention is present by repeating the very same examination format online. Retention of knowledge is higher when starting with VSA (group B) both for a subsequent SC format or a VSA format. These data would argue for more use of the VSA format at least in pharmacology examinations.