Affiliation:
1. Keck School of Medicine of the University of Southern California Los Angeles California USA
2. American Board of Emergency Medicine East Lansing Michigan USA
3. Mayo Clinic Rochester Minnesota USA
4. Wexner Medical Center at The Ohio State University Columbus Ohio USA
Abstract
AbstractBackgroundThe American Board of Emergency Medicine (ABEM) in‐person Oral Certification Examination (OCE) was halted abruptly in 2020 due to the COVID‐19 pandemic. The OCE was reconfigured to be administered in a virtual environment starting in December 2020.ObjectivesThe purpose of this investigation was to determine whether there was sufficient validity and reliability evidence to support the continued use of the ABEM virtual Oral Examination (VOE) for certification decisions.MethodsThis retrospective, descriptive study used multiple data sources to provide validity evidence and reliability data. Validity evidence focused on test content, response processes, internal structure (e.g., internal consistency and item response theory), and the consequences of testing. A multifaceted Rasch reliability coefficient was used to measure reliability. Study data were from two 2019 in‐person OCEs and the first four VOE administrations.ResultsThere were 2279 physicians who took the 2019 in‐person OCE examination and 2153 physicians who took the VOE during the study period. Among the OCE group, 92.0% agreed or strongly agreed that the cases on the examination were cases that an emergency physician should be expected to see; 91.1% of the VOE group agreed or strongly agreed. A similar pattern of responses given to a question about whether the cases on the examination were cases that they had seen. Additional evidence of validity was obtained by the use of the EM Model, the process for case development, the use of think‐aloud protocols, and similar test performance patterns (e.g., pass rates). For reliability, the Rasch reliability coefficients for the OCE and the VOE during the study period were all >0.90.ConclusionsThere was substantial validity evidence and reliability to support ongoing use of the ABEM VOE to make confident and defensible certification decisions.
Subject
Emergency Nursing,Education,Emergency Medicine
Cited by
3 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献