The Rising Quality of Randomized Controlled Trials in The Journal of Bone & Joint Surgery

Abstract

Background: Previous reports found that 40% of randomized controlled trials (RCTs) published in The Journal of Bone & Joint Surgery (JBJS) from 1988 to 2000 and 47% of those published from 2001 to 2013 were of high quality. The purpose of this study was to assess the quality of RCTs published from 2014 to 2022 in JBJS and to compare these findings with those of prior analyses in order to identify trends over time and areas for continued improvement. Methods: PubMed was searched for the term “randomized controlled trial” to identify studies published in JBJS from 2014 to 2022. Each included RCT was evaluated with use of the Detsky score and a risk-of-bias assessment modified from the Cochrane tool. These evaluations were then compared with previous evaluations of RCTs from the 1988 to 2000 and 2001 to 2013 periods with use of independent-sample t tests. A transformed Detsky score of >75% and a modified risk-of-bias score of ≥8 were defined as being indicative of high quality. Results: A total of 218 RCTs were published in JBJS from 2014 to 2022. An a priori sample size was calculated in 183 studies (83.9%). A total of 152 (83.1%) of the 183 studies enrolled the calculated number of patients, of which 126 (82.9%) maintained an adequate number at the time of final follow-up. Most RCTs were conducted at a single center (146 of 218; 67%), evaluated a surgical intervention (162 of 218; 74%), and reported positive results (142 of 218; 65%). The mean transformed Detsky score was 85% ± 10% (95% confidence interval, 83.7% to 86.3%), with 82% of trials (179 of 218) scored as high quality. The mean transformed Detsky score from 2014 to 2022 was higher than that from 1988 to 2000 and that from 2001 to 2013 (85% versus 76% and 68%, respectively; p < 0.001). The mean modified risk-of-bias score was 7 ± 1, with 42% of trials (92 of 218) scored as high quality. RCTs published from 2014 to 2022 had a higher mean modified risk-of-bias score than those published from 2001 to 2013 (7 ± 1 versus 6 ± 1; p < 0.001). Compared with the 2001 to 2013 and 2014 to 2022 periods, the 1988 to 2000 period had a greater proportion of trials that reported positive results (51% and 65% versus 82%, respectively; p < 0.001) and that included data from multiple centers (31% and 33% versus 67%; p < 0.001). Conclusions: The quality of RCTs published in JBJS from 2014 to 2022 has improved from that reported previously, as demonstrated by the increases in the modified risk-of-bias score and transformed Detsky score from prior periods. This may be the result of journal policies such as the requirements of CONSORT adherence and prospective trial registration. Investigators should focus on improving the clarity of reporting, limiting attrition bias, and making efforts to blind support staff in order to increase the quality of future RCTs. Clinical Relevance: Improving the quality of RCTs is crucial given their potential to influence current clinical practice.