The methodological quality of systematic reviews regarding the Core Outcome Set (COS) development

Abstract

Abstract Background The Core Outcome Measures in Effectiveness Trials (COMET) working group proposed core outcome sets (COS) to address the heterogeneity in outcome measures in clinical studies. According to the recommendations of COMET, performing systematic reviews (SRs) usually was the first step for COS development. However, the SRs that serve as a basis for COS are not specifically appraised by organizations such as COMET regarding their quality. Here, we investigated the status of SRs related to development of COS and evaluated their methodological quality. Methods We conducted a search on PubMed to identify SRs related to COS development published from inception to May 2022. We qualitatively summarized the disease included in SR topics, and the studies included in the SRs. We evaluated the methodological quality of the SRs using AMSTAR 2.0 and compared the overall quality of SRs with and without protocols using the Mann-Whitney U test. Results We included 175 SRs from 23 different countries or regions, and they mainly focused on five diseases: musculoskeletal system or connective tissue disease (n = 19, 10.86%), injury, poisoning, or certain other consequences of external causes (n = 18, 10.29%), digestive system disease (n = 16, 9.14%), nervous system disease (n = 15, 8.57%), and genitourinary system disease (n = 15, 8.57%). Although 88.00% of SRs included randomized controlled trials (RCTs), only a few SRs (23.38%) employed appropriate tools to assess the risk of bias in RCTs. The assessment results on the basis of AMSTAR 2.0 indicated that most SRs (93.71%) were rated as ‘’critically low’’ to ‘’low’’ in terms of overall confidence. The overall confidence of SRs with protocols was significantly higher than that without protocols (P <.001). Compared to the SRs with protocols on Core Outcome Measures in Effectiveness Trials (COMET), SRs with protocols on PROSPERO were of better overall confidence (P = .017). Conclusion The overall quality of published SRs regarding COS development was poor. Our findings emphasize the need for researchers to carefully select the disease topic and strictly adhere to the requirements of optimal methodology when conducting a SR for the establishment of a COS.