On False Augmented Agency and What Surveillance Capitalism and User-Centered Design Have to Do With It

Abstract

In the last years, there has been a surge in AI-powered products. Often marketed as "free", these services operate as hooks to lure unsuspecting users into voluntarily giving up data about every aspect of their life. Their data is the primary fuel of surveillance capitalism, a new economic system that exclusively benefits so-called Big Tech organisations at the expense of personal privacy and freedom of choice. This paper argues the ways these AI-powered products are being imagined and designed is further generalising a kind of, "enframing" that encourages a bureaucratic relationship with the world disguised as (a false sense of) augmented agency. This paper shows that technologically informed philosophical reflections can contribute to getting ourselves back into the feedback loop of technological mediation by helping us recognise our "becoming" with technologies as a design process.