Machine Learning for Detecting Trends and Topics from Research Papers and Proceedings in Biomedical Literature

Abstract

Abstract The work presents robust statistical and exploratory analysis to demonstrate the effects of performances of machine learning (ML) classifiers and sampling techniques in document datasets. 1,000 portable document format (PDF) files are divided into five labels from the World Health Organization COVID-19 Research Downloadable Articles and PubMed Central databases for positive and negative papers. PDF files are converted into unstructured raw text files and pre-processed before tokenization. Training size and subsampling were varied experimentally to determine their effect on the performance measures, such as accuracy, precision, recall, and AUC. Supervised classification is performed using the Random Forest, Naïve Bayes, Decision Tree, XGBoost, and Logistic Regression. Imbalanced sampling techniques are implemented using the Synthetic Minority Oversampling Technique, Random Oversampling, Random Undersampling, TomekLinks, and NearMiss to address the problem of distribution of positive and negative samples. R and the tidyverse are used to conduct statistical and exploratory data analysis on performance metrics. The ML classifiers achieve an average precision score of 78% and a recall score of 77%, while the sampling techniques have higher average precision and recall scores of 80% and 81%, respectively. Correcting imbalanced sampling supplied significant p-values from NearMiss, ROS, and SMOTE for precision and recall scores. This work has shown with statistical significance including the analysis of variance (ANOVA) that training size variation, subsampling, and imbalanced sampling techniques with ML algorithms can improve performances in document datasets.