Artificial intelligence in medicine – is too much transparency a good thing?

Abstract

Some encouraging uses for AI in medicine will lead to potentially novel legal liability issues. Complex algorithms involve an opacity that creates problems for the medical and legal professions alike. As iatrogenic injury is common in medical malpractice, the medical profession is understandably concerned when AI is introduced in diagnostic and therapeutic devices and events and outcome cannot be fully explained due to the “black box” effect. A concern about machine learning algorithms is the black box issue and understanding how conclusions or outcomes are reached. The deployment of AI devices in healthcare will require an increase in a clinician’s understanding of AI to increase the transparency of their use. An important aspect of medical treatment is the notion of “therapeutic privilege”. This will only arise in limited circumstances and requires the clinician to make a judgment, based on reasonable grounds, that the patient’s physical or mental health might be seriously harmed by providing the information. Given the complexity of AI and the black box effect, could too much AI transparency possibly overwhelm a patient, such that it may dissuade them from giving consent in circumstances where treatment is necessary and essential? In other words, too much AI transparency and information may inadvertently hinder treatment and progress.