A novel customizing knowledge graph evaluation method for incorporating user needs

Abstract

AbstractKnowledge graphs are now widely used in various domains, including Question-and-answer systems, intelligent search and recommendation systems, and intelligent decision-making systems. However, knowledge graphs inevitably contain inaccurate and incomplete knowledge during the creation process, which leads to a reduction in the usefulness of knowledge graphs. Therefore, to assess the usefulness of knowledge graphs based on specific application requirements, quality assessment is particularly important. Among them, accuracy assessment, as a necessary dimension, reflects the degree of correctness of the triples. However, in the actual assessment process, the existing assessment methods do not consider the user’s needs and do not implement the concept of “Fitness for Use”. Meanwhile, it takes a lot of labor cost to evaluate the accuracy of large-scale knowledge graphs. Therefore, to ensure the accuracy of the assessment in a cost-saving way while meeting the needs of users, we propose and implement a novel accuracy assessment method that focuses on the requirements of users by designing an effective sampling method to obtain accurate assessment results that are more practical and instructive for users. Finally, the performance of our proposed method is evaluated by comparing it with the real accuracy rate, and the experimental results show that the accuracy rate obtained by the proposed method is very close to the real accuracy rate, and the sample size is minimized.