The use of experimental vignettes in studying police procedural justice: a systematic review

Abstract

Abstract Objectives The current review assesses the methodological characteristics of between-subjects experiments, in particular documenting the scenarios and treatments described in each vignette, the extent to which confounds are embedded or accounted for in the design, and the analytic approach to estimating direct and interaction effects. Methods We conducted a pre-registered systematic review of 20 publications containing 20 independent studies and 23 vignette scenarios. Results We find that the majority of studies rely on non-probability convenience sampling, manipulate a combination of procedural justice elements at positive and negative extremes, but often do not address potential confounds or threats to internal validity. The procedural justice manipulations that combine different elements show relatively consistent associations with a range of attitudinal outcomes, whereas the results for manipulations that test individual components of procedural justice (e.g., voice) are more mixed. Conclusions Based on our review, we recommend that future studies using text-based vignettes disaggregate different elements of procedural justice in manipulations, and include a gradient of treatment or behavior (including control) to avoid comparing extremes, to incorporate potential confounders as either fixed covariates or manipulations, and to formally assess the information equivalence assumption using placebo tests.