Author:
Zhu Xian,Chen Yuanyuan,Gu Yueming,Xiao Zhifeng
Abstract
Recent advances have witnessed a trending application of transfer learning in a broad spectrum of natural language processing (NLP) tasks, including question answering (QA). Transfer learning allows a model to inherit domain knowledge obtained from an existing model that has been sufficiently pre-trained. In the biomedical field, most QA datasets are limited by insufficient training examples and the presence of factoid questions. This study proposes a transfer learning-based sentiment-aware model, named SentiMedQAer, for biomedical QA. The proposed method consists of a learning pipeline that utilizes BioBERT to encode text tokens with contextual and domain-specific embeddings, fine-tunes Text-to-Text Transfer Transformer (T5), and RoBERTa models to integrate sentiment information into the model, and trains an XGBoost classifier to output a confidence score to determine the final answer to the question. We validate SentiMedQAer on PubMedQA, a biomedical QA dataset with reasoning-required yes/no questions. Results show that our method outperforms the SOTA by 15.83% and a single human annotator by 5.91%.
Subject
Artificial Intelligence,Biomedical Engineering
Reference47 articles.
1. Learning Contextualized Document Representations for Healthcare Answer Retrieval
2. Biomedical question answering: a survey;Athenikos;Comput. Methods Prog. Biomed,2010
3. Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining,;Baccianella,2010
4. Boolq: exploring the surprising difficulty of natural yes/no questions;Clark;arXiv,2019
5. Electra: pre-training text encoders as discriminators rather than generators;Clark;arXiv,2020
Cited by
4 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献