Accuracy and Reliability of Peer Assessment of Athletic Training Psychomotor Laboratory Skills

Abstract

AbstractContext:Peer assessment is defined as students judging the level or quality of a fellow student's understanding. No researchers have yet demonstrated the accuracy or reliability of peer assessment in athletic training education.Objective:To determine the accuracy and reliability of peer assessment of athletic training students' psychomotor skills.Design:Cross-sectional study.Setting:Entry-level master's athletic training education program.Patients or Other Participants:First-year (n  =  5) and second-year (n  =  8) students.Main Outcome Measure(s):Participants evaluated 10 videos of a peer performing 3 psychomotor skills (middle deltoid manual muscle test, Faber test, and Slocum drawer test) on 2 separate occasions using a valid assessment tool. Accuracy of each peer-assessment score was examined through percentage correct scores. We used a generalizability study to determine how reliable athletic training students were in assessing a peer performing the aforementioned skills. Decision studies using generalizability theory demonstrated how the peer-assessment scores were affected by the number of participants and number of occasions.Results:Participants had a high percentage of correct scores: 96.84% for the middle deltoid manual muscle test, 94.83% for the Faber test, and 97.13% for the Slocum drawer test. They were not able to reliably assess a peer performing any of the psychomotor skills on only 1 occasion. However, the ϕ increased (exceeding the 0.70 minimal standard) when 2 participants assessed the skill on 3 occasions (ϕ  =  0.79) for the Faber test, with 1 participant on 2 occasions (ϕ  =  0.76) for the Slocum drawer test, and with 3 participants on 2 occasions for the middle deltoid manual muscle test (ϕ  =  0.72).Conclusions:Although students did not detect all errors, they assessed their peers with an average of 96% accuracy. Having only 1 student assess a peer performing certain psychomotor skills was less reliable than having more than 1 student assess those skills on more than 1 occasion. Peer assessment of psychomotor skills could be an important part of the learning process and a tool to supplement instructor assessment.