Baseline categorical variables in spine randomized controlled trials

Abstract

Abstract Background Recently Levayer and colleagues surveyed randomized controlled trials in 4 spine journals for signs of integrity issues based on an assessment of baseline p-values for categorical data. They concluded that there was no evidence of “systemic fraudulent behaviour”, and that data were consistent with random allocation. Methods We used their published dataset to compare reported with independently calculated baseline p-values and the observed and expected distributions of frequency counts and baseline p-values using the reappraised package for R. Results In 51/929 (5.5%) baseline variables, the sum of frequencies by level did not agree with the reported number of participants. For one third of reported p-values (172/522), we could not calculate a matching p-value using a range of standard statistical tests. Sparse data were common; for 22% (74/332) variables in which the reported p-value matched a calculated p-value from a chi-square test, the expected cells were smaller than are recommended for use of chi-square tests. There were 20-25% more two-arm trials with between-groups differences in frequency counts of 1 or 2 than expected. There were small differences between the observed and expected distributions of baseline p-values, but these were dependent on how sparse data were analysed. Conclusion Incorrectly reported p-values and incorrect usage of statistical tests are common in these spine journals. There are differences between observed and expected distributions of baseline p-values and frequency counts. Collectively, the findings do raise questions about the reliability of some RCTs in major spine journals.