Meta-Analysis of Diagnostic Studies: A Comparison of Random Intercept, Normal-Normal, and Binomial-Normal Bivariate Summary ROC Approaches

Abstract

Background . The authors compared 3 recently introduced refinements of the Littenberg and Moses summary receiver operating characteristic (ROC) method for pooling studies of a diagnostic test: the random intercept (RI) linear meta-regression model, the approximate normal distribution (normal-normal [NN] model), and the binomial distribution (binomial-normal [BN] model). Methods . Using data from a published meta-analysis of magnetic resonance imaging of the menisci and cruciate ligaments, the authors varied the overall sensitivity and specificity, the between-studies variance, the within-study sample size, and the number of studies to evaluate the performances of the 3 methods in a simulation study. The parameters to be compared are the associated intercept, slope, and residual variance, using bias, mean squared error, and coverage probabilities. Results . The BN method always gave unbiased estimates of the intercept and slope parameter. The coverage probabilities were also reasonably acceptable, unless the number of studies was very small. In contrast, the RI and NN methods could produce large biases with poor coverage probabilities, especially when sample sizes of individual studies were small or when sensitivities or specificities were close to 1. Although this was rare in the simulations, the bivariate methods can suffer from nonconvergence mostly due to the correlation being close to ± 1. Conclusion . The binomial-normal model performed better than the other recently introduced methods for meta-analysis of data from studies of test performance.