EVALUATING SUBJECTIVE ASPECTS OF HCI ON AN EXAMPLE OF A NON-TASK ORIENTED CONVERSATIONAL SYSTEM

Abstract

The evaluation of subjective aspects of HCI, such as human-likeness, likeability or users' emotions towards computers is still quite a neglected issue, especially in the field of non-task oriented conversational systems (chatterbots). In this paper we try to bridge this gap by proposing a new methodology of evaluation. The methods presented were tested in our research on humor-equipped chatterbots. We describe them in details, discuss their drawbacks and usability. In one of the presented methods we used an emotiveness analysis system, which itself can be considered an AI tool, as it was used to detect users' emotions towards conversational systems, and to perform their automatic evaluation. We also propose some methods that we have not used yet, which, however, seem applicable in this field, such as brain scanning techniques. Finally, we give some ideas that should be addressed in the future.