Vul-Mixer: Efficient and Effective Machine Learning–Assisted Software Vulnerability Detection

Abstract

Recent Machine Learning–Assisted Software Vulnerability Detection (MLAVD) research has focused on large-scale models with hundreds of millions of parameters powered by expensive attention- or graph-based architectures. Despite increased model capacity, current models have limited accuracy and struggle to generalize to unseen data. Additionally, the computational resources required to train and serve the models further reduce their usefulness. We argue this is caused by a misalignment between how human brains process code and how MLAVD models are designed. In this paper, we study resource-efficient approaches to MLAVD with the goal of maintaining or strengthening generalizability while reducing computational costs such that the model may be run on an economy developer machine. Our contributions are as follows: (1) We perform the first known study of resource-efficient MLAVD, showing such models can be competitive with strong MLAVD baselines; (2) We design Vul-Mixer, a resource-efficient architecture inspired by how the human brain processes code; and, (3) We demonstrate that Vul-Mixer is efficient and effective by maintaining 98.3% of the state-of-the-art generalization ability using only 0.2% of the parameters and 173 MB of memory.