On the Level of Measurement of Subjective Psychometric Ratings

Abstract

Subjective psychometric ratings are critical tools in human factors engineering. Despite widespread use, there is controversy about whether such ratings can be treated at cardinal levels of measurement. Answering this question is important given that a measure’s level determines what mathematical and statistical operations can be meaningfully applied to them. This paper synthesizes results from an effort (published over several papers) that developed a method for assessing the level of measurement of subjective ratings and used it to evaluate common trust, situation awareness, and workload metrics. This paper synthesizes these results to determine that subjective measures should rarely, if ever, be treated as ratio. It also found that, in a population analysis, some measures (like all the ones considered) can be treated as interval. In all other situations, and especially in analyses or modeling efforts for individuals or small sample sizes, ordinal is the safest default.