Classification of posttransfusion adverse events using a publicly available artificial intelligence system

Abstract

AbstractBackgroundCorrect classification of transfusion reactions is important not only for effective patient care and donor management but also for accurate tracking of events in hemovigilance systems. We compared the ability of a generative artificial intelligence (AI) system to correctly diagnose hypothetical clinical situations as transfusion reactions in comparison to previous studies reporting the accuracy of transfusion medicine (TM) specialists in assessing these cases.MethodsAn AI system was requested to assess 36 case scenarios to provide a diagnosis, severity, and imputability of the transfusion reactions using the CDC National Healthcare Safety Network (NHSN) criteria. Responses were compared to an expert panel's classifications and to the published responses of a panel of TM specialists. Additionally, the AI's responses were compared to the TM specialists' prior attempts to use the TrDDx web‐based algorithm for the five most challenging cases.ResultsThe AI's classification accuracy varied widely depending on the NHSN category. The AI accurately classified all transfusion‐associated circulatory overload and transfusion‐related acute lung injury cases, exceeding TM specialists' assessments. Conversely, it did not correctly identify any cases in select NHSN categories such as DSTR. Overall accuracy among all diagnostic categories was 48.7% for AI responses versus 72.1% for prior TM specialist responses (p = 0.005). AI‐generated responses included non‐standard terminology, limited severity assessments, and no imputability determinations.DiscussionA generative AI system may have a role in helping healthcare providers to consider transfusion reaction categories that might be missed, but caution is advised in applying the AI's output to transfusion reaction classification at present.