Would doctors dream of electric blood bankers? Large language model‐based artificial intelligence performs well in many aspects of transfusion medicine

Abstract

AbstractBackgroundLarge language models (LLMs) excel at answering knowledge‐based questions. Many aspects of blood banking and transfusion medicine involve no direct patient care and require only knowledge and judgment. We hypothesized that public LLMs could perform such tasks with accuracy and precision.Study Design and MethodsWe presented three sets of tasks to three publicly‐available LLMs (Bard, GPT‐3.5, and GPT‐4). The first was to review short case presentations and then decide if a red blood cell transfusion was indicated. The second task was to answer a set of consultation questions common in clinical transfusion practice. The third task was to take a multiple‐choice test experimentally validated to assess internal medicine postgraduate knowledge of transfusion practice (the BEST‐TEST).ResultsIn the first task, the area under the receiver operating characteristic curve for correct transfusion decisions was 0.65, 0.90, and 0.92, respectively for Bard, GPT‐3.5 and GPT‐4. All three models had a modest rate of acceptable responses to the consultation questions. Average scores on the BEST‐TEST were 55%, 40%, and 87%, respectively.ConclusionWhen presented with transfusion medicine tasks in natural language, publicly available LLMs demonstrated a range of ability, but GPT‐4 consistently scored very well in all tasks. Research is needed to assess the utility of LLMs in transfusion medicine practice. Transfusion Medicine physicians should consider their role alongside such technologies, and how they might be used for the benefit and safety of patients.