Do the Readability and Average Item Length of Personality Scales Affect Their Reliability?

Abstract

Using meta-analytic data, we investigated three attributes of personality scales – their readability, the average number of words per item, and whether adjectives or statements were used – to determine the effects that these qualities have on scale reliability (Cronbach’s α). From a large pool of studies examined in three prior meta-analyses, we extracted the relevant data for seven personality traits measured in 167 independent samples (N = 55,593), and obtained measures of the traits’ readability, the average number of words per item, and whether adjectives or statements were used as the item format. We found strong and consistent evidence that using fewer words per item produced higher scale reliabilities, and there was also evidence in favor of using adjectives, as opposed to statements, as the item format when measuring broad traits. In contrast, the results for readability were inconsistent and mostly nonsignificant. These results have practical implications for scale developers as well as researchers, and they raise the possibility that shorter and more “de-contextualized” items do a better job of assessing personality traits.