The inconsistency of p-curve: Testing its reliability using the power pose and HPA debates

Abstract

Recent works have called into question whether p-curve can reliably assess the presence of "evidential value" within a set of studies. To examine an as-yet unexplored issue, we examined the method used to identify p-values for inclusion in a p-curve analysis. We developed iterated p-curve analysis (IPA), which calculates and p-curves every permutation for a set of reported p-values, and applied it to the data reported in several published p-curve analyses. Specifically, we investigated two phenomena for which p-curves have been used to evaluate the presence of evidential value: the power pose and the hypothalamic-pituitary-adrenal (HPA) reactivity debates. The iterated p-curve analyses revealed that the p-curve method fails to provide reliable estimates or reproducible conclusions. We conclude that p-curve should not be used to make conclusions regarding the presence or absence of evidence for a specific phenomenon.