Affiliation:
1. University of Cagliari
2. Utrecht University
3. Maastricht University
Abstract
Existing eXplainable Artificial Intelligence (XAI) techniques support people in interpreting AI advice. However, although previous work evaluates the users’ understanding of explanations, factors influencing the decision support are largely overlooked in the literature. This article addresses this gap by studying the impact of
user uncertainty
,
AI correctness
, and the interaction between
AI uncertainty
and
explanation logic-styles
for classification tasks. We conducted two separate studies: one requesting participants to recognize handwritten digits and one to classify the sentiment of reviews. To assess the decision making, we analyzed the
task performance, agreement
with the AI suggestion, and the user’s
reliance
on the XAI interface elements. Participants make their decision relying on three pieces of information in the XAI interface (image or text instance, AI prediction, and explanation). Participants were shown one explanation style (between-participants design) according to three styles of logical reasoning (inductive, deductive, and abductive). This allowed us to study how different levels of AI uncertainty influence the effectiveness of different explanation styles. The results show that user uncertainty and AI correctness on predictions significantly affected users’ classification decisions considering the analyzed metrics. In both domains (images and text), users relied mainly on the instance to decide. Users were usually overconfident about their choices, and this evidence was more pronounced for text. Furthermore, the inductive style explanations led to overreliance on the AI advice in both domains—it was the most persuasive, even when the AI was incorrect. The abductive and deductive styles have complex effects depending on the domain and the AI uncertainty levels.
Funder
CRS4.Centro di Ricerca, Sviluppo e Studi Superiori in Sardegna for collaboration on the RIALE
Sardinia Regional Government and by Fondazione di Sardegna, ADAM
ASTRID
Publisher
Association for Computing Machinery (ACM)
Subject
Artificial Intelligence,Human-Computer Interaction
Reference92 articles.
1. A study on multimodal and interactive explanations for visual question answering;Alipour Kamran;CoRR,2020
2. Evaluating saliency map explanations for convolutional neural networks
3. Alexander Amini Wilko Schwarting Ava Soleimany and Daniela Rus. 2020. Deep Evidential Uncertainty. Retrieved March 22 2023 from https://openreview.net/forum?id=S1eSoeSYwr.
4. Vijay Arya Rachel K. E. Bellamy Pin-Yu Chen Amit Dhurandhar Michael Hind Samuel C. Hoffman Stephanie Houde et al. 2019. One explanation does not fit all: A toolkit and taxonomy of AI explainability techniques. arxiv:1909.03012 [cs.AI] (2019).
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Engineering Interactive Systems Embedding AI Technologies;Companion Proceedings of the 2023 ACM SIGCHI Symposium on Engineering Interactive Computing Systems;2023-06-27