Examining the Relationship Between the Readability and Comprehensibility of Practice Test Questions and Failure Rates on Learner's Permit Knowledge Tests

Abstract

Purpose The readability and comprehensibility of Learner's Permit Knowledge Test practice questions and the relationship with test failure rates across states and the District of Columbia were examined. Method Failure rates were obtained from department representatives. Practice test questions were extracted from drivers' manuals and department websites and examined for readability using Flesch–Kincaid Grade Level and comprehensibility using Question Understanding Aid. The influence of readability and comprehensibility on test failure rates was explored. Results The average failure rate from reporting jurisdictions was 42.76%. In total, 11 out of 28 jurisdictions reported that test takers fail more than half the time, while 25 out of 28 reported that test takers fail at least a quarter of the time. While 33.09% of the variability in failure rates could be accounted for by syntactic complexity of the questions, 54.18% could be accounted for by the reading ease. Discussion With few exceptions, test failure rates are systematically high across the United States. The current findings suggest that these tests may be inappropriately biased against individuals with lower levels of literacy and language ability. Implications for test developers and clinicians are discussed.