Interacting with Man or Machine: When Do Humans Reason Better?

Abstract

The resolution of complex problems is widely seen as the next challenge for hybrid human–artificial intelligence (AI) teams. This paper uses experiments to assess whether there is a difference in the quality of human reasoning depending on whether the humans interact with humans or algorithms. For this purpose, we design an interactive reasoning task and compare the performance of humans when paired with other humans and AI. Varying the difficulty of the task (i.e., steps of counterfactual reasoning required), we find that, for simple tasks, subjects perform much better if they play with other humans, whereas the opposite is true for difficult problems. Additional experiments in which subjects play with human experts show that the differences are driven by the knowledge that AI reasons correctly rather than that it is nonhuman. This paper was accepted by Elena Katok, special issue on the human-algorithm connection. Supplemental Material: The data files are available at https://doi.org/10.1287/mnsc.2023.03315 .