Quantifying Cooperation Between Rule-Based Hanabi Agents Using Information Theory

Abstract

With increasing abilities of artificial systems and agents, it has been proposed to design those systems to behave cooperatively towards human users. However, it is often unclear how to evaluate whether a cooperation in human-machine interaction takes place or how to even quantitatively describe such a cooperation. We have previously proposed a framework to quantify the degree of cooperation in an interaction between two agents, using novel measures from information theory. We here extend the initial evaluation of this framework by applying the proposed measure to rule-based artificial agents playing Hanabi. We show that the measure correlates with the number of points scored by the agents and that the framework allows to describe uni-directional interactions between stronger and weaker players. The proposed framework may be used to evaluate and guide HMI design towards more cooperative interactions, which is believed to lead to a more pleasant user experience.