Abstract
We explore the use case of question answering (QA) by a contact centre for 130,000 Dutch government employees in the domain of questions about human resources (HR). HR questions can be answered using personnel files or general documentation, with the latter being the focus of the current research. We created a Dutch HR QA dataset with over 300 questions in the format of the Squad 2.0 dataset, which distinguishes between answerable and unanswerable questions. We applied various BERT-based models, either directly or after finetuning on the new dataset. The F1-scores reached 0.47 for unanswerable questions and 1.0 for answerable questions depending on the topic; however, large variations in scores were observed. We conclude more data are needed to further improve the performance of this task.
Funder
P-Direkt, Ministry of the Interior and Kingdom Relations
Reference31 articles.
1. Know What You Don’t Know: Unanswerable Questions for SQuAD;Rajpurkar;arXiv,2018
2. Entity Linking: Finding Extracted Entities in a Knowledge Base
3. Retrospective reader for machine reading comprehension;Zhang;Proceedings of the AAAI Conference on Artificial Intelligence,2021
4. NewsQA: A Machine Comprehension Dataset
5. SQuAD: 100,000+ Questions for Machine Comprehension of Text;Rajpurkar;arXiv,2016