Abstract
AbstractContemporary organizations increasingly adopt conversational agents (CAs) as intelligent and natural language-based solutions for providing services and information. CAs offer new forms of personalization, speed, (cost-)effectiveness, and automation. However, despite their hype in research and practice, many organizations still fail to seize CAs’ potential because they lack knowledge of how to evaluate and improve the quality of CAs to sustain them in organizational operations. We aim to fill this knowledge gap by conducting a design science research project in which we aggregate insights from the literature and practice to derive an applicable set of quality criteria for CAs. Our article contributes to CA research and guides practitioners by providing a blueprint to structure the evaluation of CAs and to discover areas for systematic improvement.
Publisher
Springer Science and Business Media LLC
Subject
Management of Technology and Innovation,Marketing,Computer Science Applications,Economics and Econometrics,Business and International Management
Reference123 articles.
1. Alonso, A. F., Fuertes Castro, J. L., Martínez Normand, L., & Soza, H. (2009). Towards a set of measures for evaluating software agent autonomy. Mexican International Conference on Artificial Intelligence (MICAI), Guanajuato, México. https://doi.org/10.1109/MICAI.2009.15
2. Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D. (2019). Resilient chatbots: Repair strategy preferences for conversational breakdowns. Conference on human factors in computing systems (CHI), Glasgow, Scotland, UK. https://doi.org/10.1145/3290605.3300484
3. Benner, D., Elshan, E., Schöbel, S., & Janson, A. (2021). What do you mean? A review on recovery strategies to overcome conversational breakdowns of conversational agents. International Conference on Information Systems (ICIS). Austin, TX, United States
4. Bisio, I., Garibotto, C., Grattarola, A., Lavagetto, F., & Sciarrone, A. (2018). Smart and robust speaker recognition for context-aware in-vehicle applications. IEEE Transactions on Vehicular Technology, 67(9), 8808–8821. https://doi.org/10.1109/TVT.2018.2849577
5. Bittner, E. A. C., Oeste-Reiß, S., & Leimeister, J. M. (2019). Where is the bot in our team? Toward a taxonomy of design option combinations for conversational agents in collaborative work. Hawaii International Conference on System Sciences (HICSS), Hawaii, United States.