Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Socially

Abstract

Interfaces increasingly mimic human social behaviours. Beyond prototypical examples like chatbots, basic automated systems like app notifications or self-checkout machines likewise address or 'talk to' people in person-like ways. Whilst early evidence suggests social cues can enhance user experience, we lack a good understanding of when, and why, their use in interaction design may be inappropriate. We combined a qualitative survey (n=80) with experience sampling, interview, and workshop studies (n=11) to understand people's attitudes and preferences regarding how a range of automated systems talk to/at them. We thematically analysed examples of phrasings or conduct our participants disliked, their reasons, and how they would prefer to be treated instead. One category of inappropriate use we identified is when social design elements are used to manipulate user behaviour. We distinguish four such tactics: 'agents' playing on users' emotions (e.g., guilt-tripping, coaxing), being pushy, mothering users, or being passive-aggressive. Another category regards pragmatics: personal or situational factors that can make even a seemingly helpful or friendly message come across as rude, tactless, invasive, etc. These include contextual insensitivity (e.g., embarrassing users in public); expressing clearly false personalised care; or treating a user in ways they find misaligned with the system's role or the nature of their relationship. We discuss these inappropriate uses in terms of an emerging 'social' class of dark and anti-patterns. From participant suggestions, we offer recommendations for improving how interfaces treat people in interaction, including broader normative reflections on treating users respectfully.