Rating Scales in Accounting Research: The Impact of Scale Points and Labels

Abstract

ABSTRACT Rating scales are one of the most widely used tools in behavioral research. Decisions regarding scale design can have a potentially profound effect on research findings. Despite this importance, an analysis of extant literature in top accounting journals reveals a wide variety of rating scale compositions. The purpose of this paper is to experimentally investigate the impact of scale characteristics on participants' responses. Two experiments are conducted that manipulate the number of scale points and the corresponding labels to study their influence on the statistical properties of the resultant data. Results suggest that scale design impacts the statistical characteristics of response data and emphasize the importance of labeling all scale points. A scale with all points labeled effectively minimizes response bias, maximizes variance, maximizes power, and minimizes error. This analysis also suggests variance may be maximized when the scale length is set at 7 points. Although researchers commonly believe using additional scale points will maximize variance, results indicate increasing scale points beyond 7 does not increase variance. Taken together, a fully labeled 7-point scale may provide the greatest benefits to researchers. The importance of scale labels provides a significant contribution to accounting research as only 5 percent of the accounting studies reviewed have reported scales with all points labeled.