Is OR “portable” in meta-analysis? Time to consider bivariate generalized linear mixed model

Abstract

AbstractObjectivesA recent paper by Doi et al. advocated completely replacing the relative risk (RR) with the odds ratio (OR) as the effect measure used to report the association between a treatment and a binary outcome in clinical trials and meta-analyses. Besides some practical advantages of RR over OR and the well-known issue of the OR being non-collapsible, Doi et al.’s key assumption that the OR is “portable” in the meta-analysis, i.e., study-specific ORs are likely not correlated with baseline risks, was not well justified.Study designs and settingsWe summarized the Spearman’s rank correlation coefficient between study-specific OR and the baseline risk in 40,243 meta-analyses from the Cochrane Database of Systematic Reviews (CDSR).ResultsStudy-specific ORs are negatively correlated with baseline risk of disease (i.e., higher ORs tend to be observed in studies with lower baseline risks of disease) for most meta-analyses in CDSR. Using a meta-analysis comparing the effect of oral sumatriptan (100 mg) versus placebo on mitigating the acute headache at 2 hours after drug administration, we demonstrate that there is a strong negative correlation between OR (RR or RD) with the baseline risk and the conditional effects notably vary with baseline risks.ConclusionsReplacing RR or RD with OR is currently unadvisable in clinical trials and meta-analyses. It is possible that no effect measure is “portable” in a meta-analysis. In cases where portability of the effect measure is challenging to satisfy, we suggest presenting the conditional effect based on the baseline risk using a bivariate generalized linear mixed model. The bivariate generalized linear mixed model can be used to account for correlation between the effect measure and baseline disease risk. Furthermore, in addition to the overall (or marginal) effect, we recommend that investigators also report the effects conditioning on the baseline risk.What is New?Key findingsIn most meta-analyses in Cochrane Database of Systematic Reviews, there is notable negative correlation between ORs and baseline risks.When such a correlation is not negligible, the OR is not “portable” across studies with different baseline risks.When an effect measure is not “portable”, one may derive the effects conditioning on the baseline risk from a bivariate generalized linear mixed model.What this study adds to what was knownThe recommendation to replace the RR with the OR in clinical trials and meta-analyses is misguided.The OR is not a better effect summary than RR and RD in a single study or in meta-analyses; the noncollapsibility of OR can lead to misleading results in a single study and the OR is generally not portable in the meta-analysis.In addition to reporting effect measures such as the OR, RR or RD, it is also important to present the baseline risk.What is the implication and what should change now?When none of the effects are “portable” in a meta-analysis, in addition to report the overall (or marginal) effect, one should also report the effects conditioning on the baseline risk, regardless of the measure of choice.