Validation of a Global Rating Scale for Shoulder Arthroscopy: A Pilot Study

Abstract

Background Reliable, valid, feasible and objective assessment is desirable in all aspects of medical training to allow monitoring of progress and to ensure high standards. At present, no assessment model for shoulder arthroscopy exists that indisputably meets these criteria. The global rating scale (GRS) has been validated as a measure of technical performance in various surgical procedures. We have developed and aim to validate a GRS for shoulder arthroscopy (GRSSA). Materials and Methods Using the GRSSA, 13 shoulder surgeons, rated 10 videos of diagnostic shoulder arthroscopy showing different levels of proficiency. Assessments were performed remotely using an online system developed for the study; assessors were blinded to the experience of the surgeon who was performing the arthroscopy. Construct validity was assessed via Cronbach's a and inter-rater reliability was assessed via two-way analysis of variance (ANOVA). Results The results suggest good construct validity (high internal consistency of GRSSA categories: Cronbach's a = 0.89 to 0.96). Two-way ANOVA demonstrated a good discriminatory ability of the GRSSA but variability between raters ( p = 0.0002 to 0.01), suggesting a current lack of inter-rater reliability. Discussion The GRSSA appears to be a useful tool for the assessment of diagnostic arthroscopic skills and, using our online system, the assessor can be remote to the surgeon who is performing the procedure.