Responses to concerns raised about publications don’t address the concerns raised

Abstract

Recently in the Journal, Amanda Williams described her experience of raising concerns about a group of trials with “untrustworthy data”. We were inspired by the work of Williams and colleagues to examine these and other trials by the same research group. Similar to Williams, we found that the patterns of reported data differed from the patterns expected to arise from valid randomisation. We also identified a high proportion of reported baseline p-values for categorial variables that differed from independently calculated p-values. We reported these findings to the affected journals but none of the concerns were addressed and no action will be taken about the majority. Despite the large number of unresolved concerns about these trials, readers will be unaware of the issues, which seems entirely unsatisfactory.