Methodology for testing and monitoring artificial intelligence-based software for medical diagnostics

Abstract

BACKGROUND: The global amount of investment in companies developing artificial intelligence (AI)-based software technologies for medical diagnostics reached $80 million in 2016, rose to $152 million in 2017, and is expected to continue growing. While software manufacturing companies should comply with existing clinical, bioethical, legal, and methodological frameworks and standards, there is a lack of uniform national and international standards and protocols for testing and monitoring AI-based software. AIM: This objective of this study is to develop a universal methodology for testing and monitoring AI-based software for medical diagnostics, with the aim of improving its quality and implementing its integration into practical healthcare. MATERIALS AND METHODS: The research process involved an analytical phase in which a literature review was conducted on the PubMed and eLibrary databases. The practical stage included the approbation of the developed methodology within the framework of an experiment focused on the use of innovative technologies in the field of computer vision to analyze medical images and further application in the health care system of the city of Moscow. RESULTS: A methodology for testing and monitoring AI-based software for medical diagnostics has been developed, aimed at improving its quality and introducing it into practical healthcare. The methodology consists of seven stages: self-testing, functional testing, calibration testing, technological monitoring, clinical monitoring, feedback, and refinement. CONCLUSION: Distinctive features of the methodology include its cyclical stages of monitoring and software development, leading to continuous improvement of its quality, the presence of detailed requirements for the results of the software work, and the participation of doctors in software evaluation. The methodology will allow software developers to achieve significant outcomes and demonstrate achievements across various areas. It also empowers users to make informed and confident choices among software options that have passed an independent and comprehensive quality check.