Design and implementation of information extraction system for scientific literature using fine-tuned deep learning models

Abstract

This paper presents an overview of a quality scoring system that utilizes pre-trained deep neural network models. Two types of DL models, a classification and extractive question answering (EQA) models are used to implement components of the system. The abstracts of the scientific literature are classified into two groups, in-vivo and in-vitro, and a question and answering model architecture is constructed for extracting the following types of information (animal type, the number of animals, exposure dose, and signal frequency). The Bidirectional Encoder Representations of Transformers (BERT) model pre-trained with a large text corpus is used as our baseline model for classification and EQA tasks. The models are fine-tuned with 455 EMF-related research papers. In our experiments, the fine-tuned model showed improved performance on EQA tasks for the four-categories of questions compared to the baseline, and it also showed improvements on similar questions that were not used in training. This suggests the importance of retraining of deep learning model specifically in some areas requiring domain expertise such as scientific papers. However, additional research is needed on some implementation issues, in such cases where there are still multiple answers, or where there is no answer given in a context.