Abstract
AbstractModern-day demands for services often require an availability on a 24/7 basis as well as online accessibility around the globe. For this sake, personalized virtual assistants, called chatbots, are implemented. Such systems offer services, goods or information in natural language. These natural language processing (NLP) programs respond to the user in real time and offer an intuitive and simple interface to interact with. Advantages like these make them increasingly popular. Therefore, ensuring correct functionality of chatbots is of increasing importance. However, since different implementations and user behaviour result in unpredictable results, the chatbot’s input and output data are difficult to predict and classify as well. Under such circumstances, test cases can be inferred from the domain of possible inputs of a system under test (SUT). Ontologies are concepts used in AI to provide formal representations of knowledge for a specific domain. Such ontological models contain structured information that is used for test generation. On the other hand, testing of chatbots represents a challenge because of the absence of a test oracle. In this paper, both challenges are addressed by conceptualizing ontologies for input generation and output processing in form of a metamorphic testing approach. In this scenario, both concepts are applied for automated testing of chatbots. The approach is demonstrated on a real system from the tourism domain, thereby discussing the obtained results.
Funder
Osterreichische Forschungsforderungsgesellschaft
Graz University of Technology
Publisher
Springer Science and Business Media LLC
Subject
Safety, Risk, Reliability and Quality,Software
Reference48 articles.
1. Cycorp Inc. https://www.cyc.com/. Accessed 29 Jan 2020
2. Dialogflow. https://dialogflow.com/. Accessed 11 Dec 2018
3. Grammar-solver. https://github.com/bd21/Grammar-Solver. Accessed 13 Aug 2018
4. Oxford Reference. https://www.oxfordreference.com. Accessed 03 Feb 2020
5. RDF 1.1 Concepts and Abstract Syntax. https://www.w3.org/TR/rdf11-concepts/. Accessed 17 Feb 2020
Cited by
8 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. O que sabemos sobre testes em chatbots? Uma revisão sistemática da literatura;Anais do IX Workshop sobre Aspectos Sociais, Humanos e Econômicos de Software (WASHES 2024);2024-07-21
2. Mutation Testing for Task-Oriented Chatbots;Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering;2024-06-18
3. Coverage-based Strategies for the Automated Synthesis of Test Scenarios for Conversational Agents;Proceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024);2024-04-15
4. MutaBot: A Mutation Testing Approach for Chatbots;Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings;2024-04-14
5. CMBMeTest: Generation of Test Suites Using Model-Based Testing Plus Constraint Programming and Metamorphic Testing;Electronics;2023-12-19