Improve Classification of Security Bug Reports using fasttext. A Machine Learning Based Approach

Abstract

Abstract Software developers must handle security bug reports (SBRs) before they are widely disclosed, and the system becomes vulnerable to hackers. Bug tracking systems may contain many securities-related reports which are unlabelled as SBRs. Therefore, finding unlabelled SBRs is a challenge to help security engineers identify these security issues fast and accurately. Although many methods have been proposed for classifying SBRs, challenging issues remain due to selecting an accurate and high-performance classification algorithm. This motivates us to tackle the challenges faced by the state-of-the-art SBRs classification methods by selecting a high-performance machine learning algorithm. Therefore, the main goal of this paper is to automate the process of determining which bug report can be labeled as SBR through the use of machine learning techniques. We first extracted 45,940 bug reports from publicly available datasets of five software repositories (e.g., the work of Peters et al. and Shu et al.). Second, we conducted a study on the classification of SBRs using machine learning, where we built a fasttext classifier. We then examined the accuracy of using fasttext in detecting SBRs. Our results show that fasttext can identify SBRs with an average F1 score of 0.81. Furthermore, we investigated the generalizability of identifying SBRs by applying cross-project validation, and our results show that the fasttext classifier achieves an average F1 value of 0.65. Data and results are available at https://github.com/isultane/fasttext_classifications.