Validity of the UCEEM in use: How Does it Triangulate with Qualitative Data in Measuring the Effect of an Educational Intervention?

Abstract

Objectives Improving medical student placements relies on being able to reliably evaluate how students experience clinical learning environments. The Undergraduate Clinical Education Environment Measure (UCEEM) is an increasingly used validated tool designed to allow such evaluations. This study aims to further characterize how the UCEEM relates to qualitative evaluation. methods Students on placement at one hospital were invited to complete the UCEEM before and after the implementation of an innovative new placement structure. Additionally, focus groups were employed to collect qualitative data on their experiences. a novel protocol to triangulate the output of the UCEEM with the qualitative data was developed. Results The UCEEM showed good internal consistency (Cronbach's Alpha 0.79-0.91) and internal correlation. Implementation of the intervention created significant improvements in the overall UCEEM scores ( P = .008) and in the “Learning in and through work and quality of supervision” ( P = .048), “Preparedness for student entry” ( P = .033) and “Workplace interaction patterns and student inclusion” ( P = .039) domains. The triangulation of qualitative data with UCEEM output showed that the UCEEM allowed evaluation of some perceptions not reached through open questioning. However, mixed interpretations of UCEEM items by students led to the conflation of themes and challenges in deriving the meaning behind the score. This appeared to be the case for 14 of the 24 UCEEM items. Conclusion This investigation adds to the literature supporting the UCEEM as a validated tool. It also elucidates the limitations and relationships to qualitative data that investigators need to be aware of in its use.