An Application of Gibbs Sampling to Estimation in Meta-Analysis: Accounting for Publication Bias

Abstract

There is a widespread concern that published results in most disciplines are highly biased in favor of statistically significant outcomes. We propose a model to explicitly account for publication bias using a weight function that describes the probability of publication for a particular study in terms of a selection parameter. A Bayesian analysis of this model using flat priors on both the parameter of interest and the selection parameter is carried out using Gibbs sampling to calculate the posterior distributions of interest. The model is studied in detail for the case of a single observed result and then extended to provide a method for interpreting meta-analyses. We consider models in which the probability of publication for a study might depend on other characteristics of the study—in particular, the size of the study. Finally, we apply our model to a published meta-analysis which examined the effect of coaching on scores on the SAT.