Quantitative and Qualitative evaluation of the recent Artificial Intelligence in Healthcare publications using Deep-Learning

Abstract

AbstractBackgroundAn ever-increasing number of artificial intelligence (AI) models targeting healthcare applications are developed and published every day, but their use in real-world decision-making is limited. Beyond a quantitative assessment, it is important to have a qualitative evaluation of the maturity of these publications with additional details related to trends in the type of data used type of models developed across the healthcare spectrum.MethodsWe assessed the maturity of selected peer-reviewed AI publications pertinent to healthcare for the years 2019–2021. For the report, the data collection was performed by PubMed search using the Boolean operators “machine learning” OR “artificial intelligence” AND “2021”, OR “2020”, OR ‘‘2019” with the English language and human subject research as of December 31, each year. All three years selected were manually classified into 26 distinct medical specialties. We used the Bidirectional Encoder Representations from Transformers(BERT) neural networks model to identify the maturity level of research publications based on their abstracts. We further classified a mature publication based on the healthcare specialty and geographical location of the article’s senior author. Finally, we manually annotated specific details from mature publications, such as model type, data type, and disease type.ResultsOf the 7062 publications relevant toAI in healthcarefrom 2019–2021, 385 were classified as mature. In 2019, 6.01 percent of publications were mature. 7.7 percent were mature in 2020, and 1.81 percent of publications were mature in 2021. Radiology publications had the most mature model publications across all specialties over the last three years, followed by pathology in 2019, ophthalmology in 2020, and gastroenterology in 2021. Geographical pattern analysis revealed a non-uniform distribution pattern. In 2019 and 2020, the United States ranked first with a frequency of 22 and 50, followed by China with 20 and 47. In 2021, China ranked first with 17 mature articles, followed by the United States with 11 mature articles. Imaging-based data was the primary source, and deep learning was the most frequently used modeling technique in mature publications.InterpretationDespite the growing number of publications of AI models in healthcare, only a few publications have been found to be mature with a potentially positive impact on healthcare. Globally, there is an opportunity to leverage diverse datasets and models across the health spectrum, to develop more mature models and related publications, which can fully realize the potential of AI to transform healthcare.