Commercial ERP Chatbots : Conversational Intelligence Agents' Performance Analysis, User experience Benchmarks, and Quality Standards

Abstract

A key goal of this paper is to investigate commercial applications for chatbots and propose several evaluation metrics for evaluating an embodied conversational agent's performance, usability, and overall quality. Our assessment is based on metrics that include a) size of market share & reach, b) range of users and c) likelihood of being the most advanced commercial chatbot created by their creators. Our study examines several aspects of the function of embodied conversational agents, including their visual appearance, implementation of web sites, speech synthesis unit, built-in knowledge base (with general and specialized information), presentation of knowledge and additional functionality, emergency responses in unexpected situations, and user ratings. An evaluation of any commercial chatbot deployment requires a multidimensional assessment and the current state of Conversational Intelligence in the commercial virtual assistant market.