Zero-Shot Construction of Chinese Medical Knowledge Graph with GPT-3.5-turbo and GPT-4

Abstract

Knowledge graphs have revolutionized the organization and retrieval of real-world knowledge, prompting interest in automatic NLP-based approaches for extracting medical knowledge from texts. However, the availability of high-quality Chinese medical knowledge remains limited, posing challenges for constructing Chinese medical knowledge graphs. As LLMs like ChatGPT show promise in zero-shot learning for many NLP downstream tasks, their potential on constructing Chinese medical knowledge graphs is still uncertain. In this study, we create a Chinese medical knowledge graph by manually annotating textual data and using ChatGPT to automatically generate the graph. We refine the results using filtering and mapping rules to align with our schema. The manually generated graph serves as the ground truth for evaluation, and we explore different methods to enhance its accuracy through knowledge graph completion techniques. As a result, we emphasize the potential of employing ChatGPT for automated knowledge graph construction within the Chinese medical domain. While ChatGPT successfully identifies a larger number of entities, further enhancements are required to improve its performance in extracting more qualified relations.