Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report

Abstract

Summary Objective Scientific and ethical justification for new clinical trials requires them to have been designed in the light of scientifically defensible assessments of relevant previous research. Reliable interpretation of the results of new clinical trials entails setting them in the context of updates of the reviews upon which they were deemed scientifically and ethically justifiable. We have shown previously that most reports of randomized trials published in five general medical journals in May 1997 and in May 2001 failed to set their results in the context of the findings from similar research. In the current study, we assess whether there had been progress in this respect in 2005 and also investigate the extent to which reports begin by referring to systematic reviews providing the justification for the new research reported. Design Assessment of the Introduction and Discussion sections in all reports of randomized trials published during May 2005 in five general medical journals. Setting Reports of randomized trials in five general medical journals. Participants Annals of Internal Medicine, BMJ, JAMA, Lancet and New England Journal of Medicine. Interventions None. Main outcome measures The inclusion or mention of one or more systematic reviews in the Introduction or Discussion section of each report assessed. Results We found 18 reports of randomized trials. The Introduction sections referred to systematic reviews in five (27%) of these reports. None of the Discussion sections of the 15 reports of trials that were not the first published trials to address the question studied placed the results of the new trial in the context of an updated systematic review of other research. Although reference was made to relevant systematic reviews in five of these 15 reports, there was no integration—quantitative or qualitative—of the results of the new trials in an update of these reviews. In the remaining ten reports there was no evidence that any systematic attempt had been made to set the new results in the context of previous trials. Conclusions There is no evidence of progress between 1997 and 2005 in the proportion of reports of trials published in general medical journals which discussed new results within the context of up-to-date systematic reviews of relevant evidence from other controlled trials. Although the proportion of trials referring to systematic reviews in Discussion sections has increased, the majority of reports continued to fail even to do this. Similarly, most researchers appear not to have considered a systematic review when designing their trial. Researchers and journal editors do a disservice to the interests of the public and others involved in healthcare decision-making by acquiescing in this situation.