Development of an Objective Structured Clinical Examination Station for Pediatric Occupational Therapy and an Evaluation of Its Quality

Abstract

Abstract Importance: The Objective Structured Clinical Examination (OSCE) is a highly valued measure of students’ clinical competencies in medical education. However, few studies have reported on the administration of the OSCE in pediatric occupational therapy education. Objective: To describe the development of a pediatric occupational therapy OSCE station to evaluate students’ use of a standardized assessment and examine its standard setting, failure rates, and psychometric properties. Design: Prospective, cross-sectional, observational study design. Setting: Three OSCE stations in a university clinical skills center. Participants: Five experienced occupational therapists, 60 examinees, 44 child standardized patients, 44 chaperones, and 15 examiners. Measures: The sum of the rating scale and the global performance scores were used. The rating scale measured the examinee’s clinical competences in administering a standardized assessment. The 5-point global performance score was used to evaluate the examinee’s whole performance. Results: The OCSE station’s expert validity was acceptable (item-level content validity index [CVI] = 0.8–1.0; scale-level CVI = 0.98). Passing scores according to the Angoff method (passing score = 14) and the contrasting-groups M–SD method (passing score = 13) were similar. Failure rates were high (61.7%–73.3%). Internal consistency was acceptable (Cronbach’s α = .78). No significant examiner effect was found (p = .554), and interexaminer reliability was acceptable (item score = 0.58–1.00; sum of the rating scale score = 0.97; global performance score = 0.79). Conclusions and Relevance: The OSCE station for using a standardized assessment is a reliable and valid measure of students’ interpersonal communication skills and assessment skills. What This Article Adds: The OSCE for education in pediatric occupational therapy is both effective and rigorous.