Black box problem and African views of trust

Abstract

AbstractThis article draws on the thinking about trust in African scholarship to describe the problems black box clinical artificial intelligence (AI) generates in health professional-patient relationships. Notably, under the assumption of a black box problem, the view of trust as inherently relational implies that health professionals cannot explain whether and how a clinical AI incorporates a patient’s values or leverages the same (in its outputs) to honour fiduciary relations. Additionally, the African view of trust as experience-based and accepting responsibility implies that health professionals can neither be held accountable for black box clinical AI outputs that they can hardly understand nor provide material information (concerning what the clinical AI does and why). Finally, given the understanding of trust as a normative concept, health professionals cannot accept patients’ vulnerabilities, and patients cannot give the same. Given that trust will play a vital role in the global acceptance of clinical AI, future studies should research—from other positionalities—how the black box problem will challenge the relationship of trust in the medical context.