Deep learning or classical machine learning? An empirical study on line‐level software defect prediction

Abstract

AbstractBackgroundLine‐level software defect prediction (LL‐SDP) serves as a valuable tool for developers to detect defective lines with minimal human effort. Recently, GLANCE was proposed as a readily implementable baseline for assessing the efficacy of newly proposed LL‐SDP models.ProblemWhile DeepLineDP, a cutting‐edge LL‐SDP model rooted in deep learning, has demonstrated state‐of‐the‐art performance, it has not yet been compared against GLANCE.ObjectiveWe aim to empirically compare DeepLineDP with GLANCE to obtain a comprehensive understanding of how deep learning contributes to solving the LL‐SDP challenge.MethodWe compare GLANCE against DeepLineDP to assess the extent to which DeepLineDP surpasses GLANCE in predicting defective files and identifying problematic lines. In order to obtain a reliable conclusion, we use the same dataset and performance metrics utilized by DeepLineDP.ResultOur experimental findings indicate that DeepLineDP does not outperform GLANCE in LL‐SDP. This suggests that the application of deep learning, in this context, does not yield the anticipated significant improvements.ConclusionThis finding underscores the need for further research in deep learning‐based LL‐SDP to attain the state‐of‐the‐art performance that remains elusive for less advanced techniques.