The Effect of Communicating AI Confidence on Human Decision Making When Performing a Binary Decision Task

Abstract

In this study, we experimentally analyzed whether people can perceive the accuracy or smartness of AI judgments, and whether the judgment accuracy of AI and the level of confidence in those judgments affect people’s decision-making. The results showed that people may perceive an AI’s smartness even when it only presents information on the results of its judgments. The results also suggest that AI accuracy and confidence affect human decision-making, and that the magnitude of the effect of AI confidence varies with AI accuracy. We also found that when a person’s ability to make a decision is less than or equal to the AI’s ability to make a decision, the human performance in a binary decision task improves regardless of AI accuracy. The results obtained in this study are similar in some respects to relationships in which people make decisions while interacting, and the findings from research on human interactions may apply to the research and development of human–AI interactions.