Malaria RDT (mRDT) interpretation accuracy by frontline health workers compared to AI in Kano State, Nigeria

Abstract

Background Although malaria is preventable and treatable, it continues to be a significant cause of illness and death. One strategy found to mitigate this is early diagnosis through testing. Due to their ease of use, sensitivity, and rapid results, malaria RDTs (mRDTs) have become the preferred diagnostic test. However, misadministration and misinterpretation errors remain a concern. This study investigated whether RDT use could be paired with a mobile application to improve the accuracy of mRDT interpretations amongst Frontline Health Care Workers (FHWs) in Kano State, Nigeria. Methods The study performed a comparative analysis of mRDT interpretations by FHWs, trained mRDT reviewers (Panel Read), and AI based computer vision technology. We specifically compared the accuracy of 1) AI algorithms’ interpretation vs. Panel Read interpretation; 2) FHW interpretation vs. Panel Read interpretation; 3) FHW interpretation vs. AI algorithms’ interpretation; and 4) AI performance for faint positives. Results Comparing the AI interpretation to Panel Read the AI performed well, correctly interpreting positives 96.38% and negatives 97.12% of the time. Interpretation accuracy of the FHWs was determined by comparing FHW interpretation to the Panel Read, finding agreement 96.82% on positives and 94.31% on negatives. The FHW interpretations were then compared to the AI, with 97.52% agreement on interpretation of positives and 93.38% agreement on negatives. Overall accuracy yielded a 96.4 weighted F1 score for the AI compared to 95.3 for FHWs. AI algorithms were additionally able to accurately classify 90.2% of the 163 mRDTs that showed a faint positive line, compared to only 76.1% for the FHWs. Conclusion The AI performed as well as experienced and trained FHWs and performed even better than FHWs on faint lines. Therefore, AI computer vision technology can assist FHWs in accurately interpreting mRDTs and reporting results in highly malaria-endemic, low-resource settings, ensuring precise reporting of mRDT results.