Methodological Quality of Retrospective Observational Studies Investigating Effects of Diabetes Monitoring Systems: a Systematic Review

Abstract

Abstract Background Retrospective observational studies (ROSs) have been frequently used to investigate treatment effects of diabetes monitoring systems (DMS), i.e. medical devices to monitor blood glucose. However, due to quality concerns, the findings of such studies were often questioned by clinical, regulatory, or health technology assessment decision-makers. We aimed to conduct a systematic review to assess the methodological quality of ROSs investigating DMS effects, and to explore the trend in quality change over time. Methods Embase, PubMed, Web of Science, and Scopus were systematically searched for English-language articles published from January 2012 to March 2021. Randomized controlled trials or other prospective studies were manually excluded. The ROBINS-I (Risk Of Bias In Non-randomized Studies – of Interventions) was used for assessing RoB. To investigate the quality change over time, we divided the study into three subgroups according to publication year, and compared the proportion of studies with the same quality level among the three subgroups. Results We identified 4926 articles, of which 72 were eligible for inclusion. Twenty-six studies were published before 2018, 22 in 2018 or 2019, and 24 after 2019. The overall methodological quality was quite low, as 61 (85%) studies were graded as facing critical or serious RoB. Also, the overall methodological quality did not substantially improve over time. The major contributors to low quality included confounding, missing data, and selection of the reported results. Conclusions The retrospective observational studies investigating DMS effects generally had a high risk of bias, and this did not substantially improve in the past ten years. Thus, clinical, regulatory, or HTA decision-makers may need strategies to effectively exploit these suboptimal studies. Also, to further improve study quality, extra efforts may be needed, such as guiding the tool selection regarding quality improvement in the tools.