Data uncertainty in 11 papers on women’s health

Abstract

Abstract Detailed analyses of published trials conducted as part of a Cochrane review led to concerns over two trials conducted by a single author. The Cochrane team therefore requested a forensic analysis of all his published papers. Eleven papers were found. Where appropriate, pairwise comparisons were made of values in baseline and outcome tables, and p-values recalculated. The distributions of baseline characteristics were assessed for compatibility with properly conducted randomization using Monte Carlo analysis and unusual features noted. The analyses brought up several areas of concern; for instance, a high degree of identical or highly similar values in some baseline and outcome tables between studies; all recalculated p-values, save for one, are different from the p-values given; for all eleven studies, the probability that participants have been grouped according to properly randomised processes is very low. In correspondence with the author, he agreed that the sample we sent him showed that published p-values were incorrect, but was unable to provide original data to conduct further checks. We conclude that analyses of papers published by this author suggest considerable data concerns and that they should not be used to inform clinical practice until further investigation is completed.