Revisiting performance appraisal measurement: Construction and validation of Rater Bias Scale

Abstract

Given the importance of the performance appraisal system and despite continued discussions and efforts to ensure the quality of supervisor rating in the appraisal system, rater effects are still pervasive and continue to undermine the validity and credibility of the appraisal system. Purpose. Thisstudy aimed to develop and validate Rater Bias Scale for the detection and control of rater effects in performance appraisal system. Method. Extensive review of literature was conducted. Separate datawere collected for exploratory and confirmatory factor analyses. Samples consisted of diverse academics and professional managers who have been involved in performance ratings in their career. Findings. The exploratory and confirmatory factor analyses conducted on the data revealed eight major dimensions of rater bias with 24 items, three items for each dimension. The scale demonstrates high content reliabilityand convergent and discriminant validities. Value of the results. Using configural and metric invariance, as well as chi-square and one-way ANOVA, the scale displays conceptual equivalence across diverse group. Implications for research and practice. The study has important implications for detecting and controlling supervisor errors in performance appraisal for research and practice. The study contributes to the literature and practice of human resource management in organisation. Limitations of the studyare highlighted and potential problem in its administration are discussed.