A rapid review exploring the effectiveness of artificial intelligence for cancer diagnosis

Author:

Wale Alesha,Shaw Hannah,Ayres Toby,Okolie Chukwudi,Morgan Helen,Everitt Jordan,Little Kirsty,Edwards Rhiannon Tudor,Davies Jacob,Lewis Ruth,Cooper Alison,Edwards Adrian

Abstract

There is growing demand for diagnostic services in the UK. This rapid review aimed to assess the effectiveness of artificial intelligence (AI) in diagnostic radiology with a focus on cancer diagnosis. A range of AI models including machine learning, deep learning and ensemble models, were assessed in this review.The review included an initial broad mapping exercise and a more in-depth synthesis of a specific sub-set of the evidence. The review included evidence available from 2018 until June 2023.A total of 92 comparative primary studies were included in the evidence map. The evidence map identified 52 studies in which the AI models were in the early stages of development and validation, and highlighted breast, lung and prostate cancers as the type of cancers most frequently reported on. 28 studies evaluating an established model and focusing on the diagnosis of breast, lung, and prostate cancer were included in the in-depth synthesis. All studies included in the in-depth synthesis were classified as diagnostic accuracy studies. Only one study evaluated an AI model that was commercially available in the UK.Most studies reported results in favour of the AI models, however, these improvements were not always statistically significant. The studies also varied considerably in terms of AI models studied, type of cancer, images used, and comparison made; and were limited in terms of their methodology. When used as a standalone diagnostic tool, there is evidence to suggest that AI can improve diagnostic accuracy or is comparable to experienced radiologists, however this may be dependent on the AI model being used. There is evidence to suggest that AI may be beneficial when used as a support tool for clinicians/radiologists with less experience. The impact of AI on the timeline involved in diagnosis appeared inconsistent. AI may speed up the diagnostic timeline when the level of cancer suspicion is low but may increase diagnostic timelines when the level of cancer suspicion is high. The evidence suggests that clinicians are accepting of AI-based assistance for cancer diagnosis.Policy and practice implicationsThe overall evidence for effectiveness appeared in favour of AI and several factors were identified that impact the effectiveness of the AI models. AI may improve diagnostic accuracy in clinicians/radiologists with less experience of interpreting radiological images. However, further well-designed high-quality research is needed from the UK and similar countries to better understand the effectiveness of AI in cancer diagnosis.Economic considerationsThere is little evidence on the cost-effectiveness of using AI for cancer diagnosis. In theory, it might be possible for AI to assist with earlier diagnosis of cancer with both health and economic benefits.Funding statementThe Public Health Wales Observatory was funded for this work by the Health and Care Research Wales Evidence Centre, itself funded by Health and Care Research Wales on behalf of Welsh Government.EXECUTIVE SUMMARYWhat is a Rapid Review?Our rapid reviews (RR) use a variation of the systematic review approach, abbreviating or omitting some components to generate the evidence to inform stakeholders promptly whilst maintaining attention to bias.Who is this Rapid Review for?The review question was suggested by the Health Sciences Directorate (Policy).Background / Aim of Rapid ReviewThere is growing demand for diagnostic services in the UK. The use of artificial intelligence in diagnosis is part of the Welsh Government’s programme for transforming and modernising planned care and reducing waiting lists in Wales. This rapid review aimed to assess the effectiveness of artificial intelligence (AI) in diagnostic radiology with a focus on cancer diagnosis. A range of AI models including machine learning, deep learning and ensemble models, were assessed in this review. The term ‘AI models’ was therefore used to encompass these different types of AI models described in the literature. The review included an initial broad mapping exercise and a more in-depth synthesis of a specific sub-set of the evidence. The focus of the in-depth synthesis was informed by the review’s stakeholders based on the findings of the mapping exercise.ResultsRecency of the evidence baseThe review included evidence available from 2018 until June 2023.Extent of the evidence baseA total of 92 comparative primary studies were included in the evidence map.The evidence map identified 52 studies in which the AI models were in the early stages of development and validation, and highlighted breast, lung and prostate cancers as the type of cancers most frequently reported on.28 studies evaluating an established model and focusing on the diagnosis of breast (n=14), lung (n=7) and prostate (n=7) cancer were includedin the in-depth synthesis.Studies included in the in-depth synthesis were conducted in the USA (n=8), Japan (n=5), UK (n=2), Italy (n=2), Turkey (n=2), Germany (n=2), Netherlands (n=2), Portugal (n=1), Greece (n=1) and Norway (n=1). Two studies were conducted across multiple countries.All studies included in the in-depth synthesis were classified asdiagnostic accuracy studies.Only one study evaluated an AI model that was commercially available in the UK.A total of 14 studies compared AI models to human readers or to other diagnostic methods used in practice, 13 studies compared the impact of AI on human interpretation of radiologic images when diagnosing cancer, four studies compared multiple AI models, and one study compared an inexperienced AI-assisted reader with an experienced reader without AI.Five studies reported on the impact of AI on diagnostic timelines (time to diagnosis, assessment time, evaluation times, and reading time).Four studies also reported on the impact of AI on inter/intra-reader variability, reliability, and agreement.One studyreported onclinicians’ acceptance and receptivenessof the use of AI for cancer diagnosis.Key findings and certainty of the evidenceMost studies reported results in favour of the AI models, however, these improvements were not always statistically significant. The studies also varied considerably in terms of AI models studied, type of cancer, images used, and comparison made; and were limited in terms of their methodology (unclear level of certainty).When used as a standalone diagnostic tool, there is evidence to suggest that AI can improve diagnostic accuracy or is comparable to experienced radiologists, however this may be dependent on the AI model being used (unclear level of certainty).There is evidence to suggest that AI may be beneficial when used as a support tool for clinicians/radiologists with less experience (unclear level of certainty).The impact of AI on the timeline involved in diagnosis appeared inconsistent. AI may speed up the diagnostic timeline when the level of cancer suspicion is low but may increase diagnostic timelines when the level of cancer suspicion is high (low level of certainty).The evidence suggests that clinicians are accepting of AI-based assistance for cancer diagnosis (low level of certainty).Research Implications and Evidence GapsNo study reported on any patient outcomes, including patient harms.No study reported on any economic outcomes.No study reported on equity outcomes, including equity of access.Further research in a real-world setting is needed to better understand the cost implications and impact on patient safety of AI for cancer diagnosis.Policy and Practice ImplicationsThe overall evidence for effectiveness appeared in favour of AI and several factors were identified that impact the effectiveness of the AI models.AI may improve diagnostic accuracy in clinicians/radiologists with less experience of interpreting radiological images.AI models are continually being developed and updated and findings are likely to vary between different AI models.Further well-designed high-quality research is needed from the UK and similar countries to better understand the effectiveness of AI in cancer diagnosis.Economic considerationsIn theory it might be possible for AI to assist with earlier diagnosis of cancer with both health and economic benefits.There is little evidence on the cost-effectiveness of using AI for cancer diagnosis. One modelling paper from the United States (US) suggests using AI in lung cancer screening using low-dose computerised tomography (CT) scans can be cost-effective, up to a cost of $1,240 per patient screened.The UK (and its constituent countries) perform consistently poorly against European and international comparators in terms of cancer survival rates. Cancer screening was suspended and routine diagnostic work deferred in the UK during the COVID-19 pandemic.The cost of cancer to the UK economy in 2019 was estimated to be least £1.4 billion a year in lost wages and benefits alone. When widening the perspective to include mortality, this figure rises to £7.6 billion a year. Pro-rating both figures to the Welsh economy and adjusting for inflation gives figures of £79 million and £429 million per annum respectively

Publisher

Cold Spring Harbor Laboratory

Reference24 articles.

1. Progress in cancer survival, mortality, and incidence in seven high-income countries 1995–2014 (ICBP SURVMARK-2): a population-based study

2. Bank of England Inflation Calculator (2023). Available at: https://www.bankofengland.co.uk/monetary-policy/inflation/inflation-calculator, Accessed 16/10/2023.

3. Cancer Research UK. (2023) Available at: https://www.cancerresearchuk.org/health-professional/cancer-statistics-for-the-uk#heading-Three Accessed 16/10/2023.

4. Department of Health and Social care (2021) £36 million boost for AI technologies to revolutionise NHS care. Available at: £36 million boost for AI technologies to revolutionise NHS care - GOV.UK (www.gov.uk) [Accessed 19th May 2023].

5. Department of Health and Social care (2023) Thousands of patients to benefit from quicker diagnosis and more accurate tests from ground-breaking AI research. Available at: Thousands of patients to benefit from quicker diagnosis and more accurate tests from ground-breaking AI research - GOV.UK (www.gov.uk) [Accessed 19th May 2023].

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3