Can artificial intelligence help decision makers navigate the growing body of systematic review evidence? A cross-sectional survey

Abstract

Abstract Background Systematic reviews (SRs) are being published at an accelerated rate. Decision makers may struggle with comparing and choosing between multiple SRs on the same topic. We aimed to understand how healthcare decision makers (e.g., practitioners, policymakers, researchers) use SRs to inform decision making, and to explore the role of a proposed AI tool to assist in critical appraisal and choosing amongst SRs.Methods We developed a survey with 21 open and closed questions. We followed a knowledge translation plan to disseminate the survey through social media and professional networks.Results Of the 684 respondents, 58.2% identified as researchers, 37.1% as practitioners, 19.2% as students, and 13.5% as policymakers. Respondents frequently sought out SRs (97.1%) as a source of evidence to inform decision making. They frequently (97.9%) found more than one SR on a given topic of interest to them. Just over half (50.8%) struggled to choose the most trustworthy SR amongst multiple. These difficulties related to lack of time (55.2%), or difficulties comparing due to varying methodological quality of SRs (54.2%), differences in results and conclusions (49.7%), or variation in the included studies (44.6%). Respondents compared SRs based on the relevance to their question of interest, methodological quality, recency of the SR search. Most respondents (87.0%) were interested in an AI tool to help appraise and compare SRs.Conclusions Respondents often sought out SRs as a source of evidence in their decision making, and often encountered more than one SR on a given topic of interest. Many decision makers struggled to choose the most trustworthy SR amongst multiple, related to a lack of time and difficulty comparing SRs varying in methodological quality. An AI tool to facilitate comparison of the relevance of SRs, the search, and methodological quality, would help users efficiently choose amongst SRs and make healthcare decisions.