Affiliation:
1. Department of Psychology, Virginia Commonwealth University
Abstract
Experimental manipulations in social psychology must exhibit construct validity by influencing their intended psychological constructs. Yet how do experimenters in social psychology attempt to establish the construct validity of their manipulations? Following a preregistered plan, we coded 348 experimental manipulations from the 2017 issues of the Journal of Personality and Social Psychology. Representing a reliance on “on-the-fly” experimentation, the vast majority of these manipulations were created ad hoc for a given study and were not previously validated before implementation. A minority of manipulations had their construct validity evaluated by pilot testing before implementation or via a manipulation check. Of the manipulation checks administered, most were face valid, single-item self-reports, and only a few met criteria for “true” validation. In aggregate, roughly two fifths of manipulations relied solely on face validity. To the extent that they are representative of the field, these results suggest that best practices for validating manipulations are not commonplace—a potential contributor to replicability issues. These issues can be remedied by validating manipulations before implementation using validated manipulation checks, standardizing manipulation protocols, estimating the size and duration of manipulations’ effects, and estimating each manipulation’s effects on multiple constructs within the target nomological network.
Funder
National Institute on Alcohol Abuse and Alcoholism
Cited by
42 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献