The Influence of Disclosing the AI Potential Error to the User on the Efficiency of User–AI Collaboration

Abstract

User–AI collaboration is an increasingly common paradigm in assistive technologies. However, designers of such systems do not know whether communicating the AI’s accuracy is beneficial. Disclosing the accuracy could lead to more informed decision making or reduced trust in the AI. In the context of assistive technologies, understanding how design decisions affect User–AI collaboration is critical because less efficient User–AI collaboration may drastically lower the quality of life. To address this knowledge gap, we conducted a VR study in which a simulated AI predicted the user’s intended action in a selection task. Fifteen participants had to either intervene or delegate the decision to the AI. We compared participants’ behaviors with and without the disclosure of details on the AI’s accuracy prior to the system’s deployment while also varying the risk level in terms of decision consequences. The results showed that communicating potential errors shortened the decision-making time and allowed the users to develop a more efficient strategy for intervening in the decision. This work enables more effective designs of the interfaces for assistive technologies using AI.