When Shooting for the Stars Becomes Aiming for Asterisks: P-Hacking in Family Business Research

Abstract

As a side-effect of increasing publication pressures, academics may be tempted to engage in p-hacking: a questionable research practice involving the iterative and incompletely-disclosed adjustment of data collection, analysis, and/or reporting, until nonsignificant results turn significant. Prior studies in entrepreneurship-related disciplines carry the implicit notion that p-hacking is predominantly an issue in top-tier journals, where incentives to do so may be highest. This study investigates p-hacking in the family business literature, a research field with roots in the broader entrepreneurship and small business literatures, and in which discourse increasingly takes place in both dedicated field journals and in the top-tier outlets in entrepreneurship and management. Analyses of p-values published in these field- and top-tier journals allow for a comparison of the prevalence and correlates of p-hacking at these different levels of prestige. The findings suggest that p-hacking is an issue of substantial—and statistically indistinguishable—magnitudes in both field- and top-tier journals. We further observe negative correlations of female authorship and employer prestige with p-hacking, where the latter is stronger in field versus top-tier journals. Implications of these findings, their limitations, and some suggestions going forward are discussed, with particular attention for the promise of preregistration and registered reports.