Generalizability Assessment of AI Models Across Hospitals: A Comparative Study in Low-Middle Income and High Income Countries

Abstract

AbstractThe integration of artificial intelligence (AI) into healthcare systems within low-middle income countries (LMICs) has emerged as a central focus for various initiatives aiming to improve healthcare access and delivery quality. In contrast to high-income countries (HICs), which often possess the resources and infrastructure to adopt innovative healthcare technologies, LMICs confront resource limitations such as insufficient funding, outdated infrastructure, limited digital data, and a shortage of technical expertise. Consequently, many algorithms initially trained on data from non-LMIC settings are now being employed in LMIC contexts. However, the effectiveness of these systems in LMICs can be compromised when the unique local contexts and requirements are not adequately considered. In this study, we evaluate the feasibility of utilizing models developed in the United Kingdom (a HIC) within hospitals in Vietnam (a LMIC). Consequently, we present and discuss practical methodologies aimed at improving model performance, emphasizing the critical importance of tailoring solutions to the distinct healthcare systems found in LMICs. Our findings emphasize the necessity for collaborative initiatives and solutions that are sensitive to the local context in order to effectively tackle the healthcare challenges that are unique to these regions.