Are We Correcting Correctly?

Abstract

The formulae for attenuation correction in meta-analysis treat reliabilities as if they were independent of each other. The current study puts this assumption of independence to the test by empirically examining the correlation among predictor and criterion reliability estimates across studies. Interdependence of reliabilities would result in either overestimation or underestimation of population correlations depending on the direction of the relationship between the reliabilities. We conducted two studies to examine the extent to which predictor and criterion reliabilities correlate across studies. Study 1 is based on 628 pairs of reliability estimates from 518 studies published in the Academy of Management Journal and the Journal of Applied Psychology between 2004 and 2011, while Study 2 is based on 564 pairs of reliability estimates from 347 studies included in a meta-analysis on perceived organizational support (POS) and some of its antecedents and outcomes. The findings in both studies show substantial correlations between predictor and criterion reliability coefficients across studies. Our article discusses important implications from these findings for future research and for the future conduct of meta-analyses.