The Inverse Data Law: Market Imperatives, Data, and Quality in AI Supported Care

Abstract

Over the last decade, the explosion of “Big Data” and its fusion with AI has led many to believe that the development and integration of AI systems in healthcare will usher in a transformative revolution that democratises access to high quality healthcare and collectively improve patient outcomes. However, the nature of market forces in the evolving data economy, has started to show evidence that the opposite is more likely to be true. This paper argues that there is a poorly understood “Inverse Data Law” that will exacerbate the widening health divide between affluent and marginalised communities because: (1) data used to train AI systems favour individuals that are already engaged with healthcare, who have the lowest burden of disease, but the highest purchasing power; and (2) data used to drive market decisions around investment in AI health technology favours tools that increase the commodification of healthcare through over-testing, over-diagnosis, and the acute and episodic management of disease, over tools that support the patient to prevent disease. This dangerous combination is more likely to cripple efforts towards preventative medicine, as data collection and utilisation tends to be inversely proportional to the needs of the patients served – the inverse data law. The paper concludes by introducing important methodological considerations in the design and evaluation of AI systems to promote systems improvement for marginalised users.