Experimental evidence of effective human–AI collaboration in medical decision-making

Abstract

AbstractArtificial Intelligence (ai) systems are precious support for decision-making, with many applications also in the medical domain. The interaction betweenmds andaienjoys a renewed interest following the increased possibilities of deep learning devices. However, we still have limited evidence-based knowledge of the context, design, and psychological mechanisms that craft an optimal human–aicollaboration. In this multicentric study, 21 endoscopists reviewed 504 videos of lesions prospectively acquired from real colonoscopies. They were asked to provide an optical diagnosis with and without the assistance of anaisupport system. Endoscopists were influenced byai($$\textsc {or}=3.05$$OR=3.05), but not erratically: they followed theaiadvice more when it was correct ($$\textsc {or}=3.48$$OR=3.48) than incorrect ($$\textsc {or}=1.85$$OR=1.85). Endoscopists achieved this outcome through a weighted integration of their and theaiopinions, considering the case-by-case estimations of the two reliabilities. This Bayesian-like rational behavior allowed the human–aihybrid team to outperform both agents taken alone. We discuss the features of the human–aiinteraction that determined this favorable outcome.