Units of Analysis and the Psychometrics of Environmental Assessment Scales

Abstract

The ecological fallacy involves interpreting results based on ecological entities, such as environmental settings, as applying to individuals. A less familiar error, the individual differences fallacy, involves interpreting results based on individuals as applying to settings. Although this second error has been quite common, little is known about the empirical consequences of using different units of analysis. This study examined the psychometrics of environmental scales when the units were individuals, observed settings, and "artificial" or random settings. Results confirm that settings rather than individuals are the appropriate units of analysis. However, methodological changes will be required beyond use of settings as units. In this study, standard reliability estimates based on settings were equivocal and only a special environmental index, the split-sample correlation, appeared adequate. Thus techniques specific to the problems of environmental research are needed.