Does Thirty-Minute Standardised Training Improve the Inter-Observer Reliability of the Horse Grimace Scale (HGS)? A Case Study

Abstract

The Horse Grimace Scale (HGS) is a facial-expression-based pain coding system that enables a range of acute painful conditions in horses to be effectively identified. Using valid assessment methods to identify pain in horses is of a clear importance; however, the reliability of the assessment is highly dependent on the assessors’ ability to use it. Training of new assessors plays a critical role in underpinning reliability. The aim of the study was to evaluate whether a 30-minute standardised training program on HGS is effective at improving the agreement between observers with no horse experience and when compared to an HGS expert. Two hundred and six undergraduate students with no horse experience were recruited. Prior to any training, observers were asked to score 10 pictures of horse faces using the six Facial Action Units (FAUs) of the HGS. Then, an HGS expert provided a 30-minute face-to-face training session, including detailed descriptions and example pictures of each FAU. After training, observers scored 10 different pictures. Cohen’s k coefficient was used to determine inter-observer reliability between each observer and the expert; a paired-sample t-test was conducted to determine differences in agreement pre- and post-training. Pre-training, Cohen’s k ranged from 0.20 for tension above the eye area to 0.68 for stiffly backwards ears. Post-training, the reliability for stiffly backwards ears and orbital tightening significantly increased, reaching Cohen’s k values of 0.90 and 0.91 respectively (paired-sample t-test; p < 0.001). The results suggest that this 30-minute face-to-face training session was not sufficient to allow observers without horse experience to effectively apply HGS. However, this standardised training program could represent a starting point for a more comprehensive training program for those without horse experience in order to increase their reliably in applying HGS.