High-cited favorable studies for COVID-19 treatments ineffective in large trials

Abstract

ABSTRACTImportanceCOVID-19 has resulted in massive production, publication and wide dissemination of clinical studies trying to identify effective treatments. However, several widely touted treatments failed to show effectiveness in large well-done randomized controlled trials (RCTs).ObjectiveTo evaluate for COVID-19 treatments that showed no benefits in subsequent large RCTs how many of their most-cited clinical studies had declared favorable results for these interventions.MethodsScopus (last update December 23, 2021) identified articles on lopinavir-ritonavir, hydroxycholoroquine/azithromycin, remdesivir, convalescent plasma, colchicine or interferon (index interventions) that represented clinical trials and that had received >150 citations. Their conclusions were assessed and correlated with study design features. The ten most recent citations for the most-cited article on each index intervention were examined on whether they were critical to the highly-cited study. Altmetric scores were also obtained.Findings40 articles of clinical studies on these index interventions had received >150 citations (7 exceeded 1,000 citations). 20/40 (50%) had favorable conclusions and 4 were equivocal. Highly-cited articles with favorable conclusions were rarely RCTs while those without favorable conclusions were mostly RCTs (3/20 vs 15/20, p=0.0003). Only 1 RCT with favorable conclusions had sample size >160. Citation counts correlated strongly with Altmetric scores, in particular news items. Only 9 (15%) of 60 recent citations to the most highly-cited studies with favorable or equivocal conclusions were critical to the highly-cited study.ConclusionMany clinical studies with favorable conclusions for largely ineffective COVID-19 treatments are uncritically heavily cited and disseminated. Early observational studies and small randomized trials may cause spurious claims of effectiveness that get perpetuated.