Sensitivity analysis with iterative outlier detection for systematic reviews and meta‐analyses

Abstract

Meta‐analysis is a widely used tool for synthesizing results from multiple studies. The collected studies are deemed heterogeneous when they do not share a common underlying effect size; thus, the factors attributable to the heterogeneity need to be carefully considered. A critical problem in meta‐analyses and systematic reviews is that outlying studies are frequently included, which can lead to invalid conclusions and affect the robustness of decision‐making. Outliers may be caused by several factors such as study selection criteria, low study quality, small‐study effects, and so on. Although outlier detection is well‐studied in the statistical community, limited attention has been paid to meta‐analysis. The conventional outlier detection method in meta‐analysis is based on a leave‐one‐study‐out procedure. However, when calculating a potentially outlying study's deviation, other outliers could substantially impact its result. This article proposes an iterative method to detect potential outliers, which reduces such an impact that could confound the detection. Furthermore, we adopt bagging to provide valid inference for sensitivity analyses of excluding outliers. Based on simulation studies, the proposed iterative method yields smaller bias and heterogeneity after performing a sensitivity analysis to remove the identified outliers. It also provides higher accuracy on outlier detection. Two case studies are used to illustrate the proposed method's real‐world performance.