Magnitudes of Experimental Effects in Social Science Research

Abstract

How the magnitude of an experimental effect may be measured has been a matter of concern for at least two decades. The phenomenon of effect size is still not well under stood, and it cannot be inferred from statistical significance. In recent years various ways of assessing the amount of variance accounted for have been proposed as measures of magnitude of effect. Other writers have proposed rulesfor standardizing effect size, with the interpretations of the measures depending largely on intuitions buttressed by some further general empirical norms. All the methods of assessing effect size have serious flaws that limit their usefulness. The various statistical procedures for estimating variance accounted for are based on different statistical models and can produce rather sharply differing results, depending on the model employed. All the methods suffer from the limitation that they reflect to too great an extent the particular characteristics of the study being reported and hence have limited generalizability.