Affiliation:
1. University of Luxembourg, Luxembourg
2. Software Engineering Application Technology Lab, Huawei, China
Abstract
How do we know a generated patch is correct? This is a key challenging question that automated program repair (APR) systems struggle to address given the incompleteness of available test suites. Our intuition is that we can triage correct patches by checking whether each generated patch implements code changes (i.e., behavior) that are relevant to the bug it addresses. Such a bug is commonly specified by a failing test case. Towards predicting patch correctness in APR, we propose a novel yet simple hypothesis on how the link between the patch behavior and failing test specifications can be drawn:
similar failing test cases should require similar patches
. We then propose
BATS
, an unsupervised learning-based approach to predict patch correctness by checking patch
B
ehavior
A
gainst failing
T
est
S
pecification.
BATS
exploits deep representation learning models for code and patches: For a given failing test case, the yielded embedding is used to compute similarity metrics in the search for historical similar test cases to identify the associated applied patches, which are then used as a proxy for assessing the correctness of the APR-generated patches. Experimentally, we first validate our hypothesis by assessing whether ground-truth developer patches cluster together in the same way that their associated failing test cases are clustered. Then, after collecting a large dataset of 1,278 plausible patches (written by developers or generated by 32 APR tools), we use
BATS
to predict correct patches:
BATS
achieves AUC between 0.557 to 0.718 and recall between 0.562 and 0.854 in identifying correct patches. Our approach outperforms state-of-the-art techniques for identifying correct patches without the need for large labeled patch datasets—as is the case with machine learning-based approaches. While
BATS
is constrained by the availability of similar test cases, we show that it can still be complementary to existing approaches: When combined with a recent approach that relies on supervised learning,
BATS
improves the overall recall in detecting correct patches. We finally show that
BATS
is complementary to the state-of-the-art PATCH-SIM dynamic approach for identifying correct patches generated by APR tools.
Funder
European Research Council
European Union’s Horizon 2020
National Natural Science Foundation of China
Natural Science Foundation of Jiangsu Province
Open Project Program of the State Key Laboratory of Mathematical Engineering and Advanced Computing
Publisher
Association for Computing Machinery (ACM)
Reference93 articles.
1. code2seq: Generating sequences from structured representations of code;Alon Uri;arXiv preprint arXiv:1808.01400,2018
2. code2vec: learning distributed representations of code
3. David Arthur and Sergei Vassilvitskii. 2006. k-means++: The Advantages of Careful Seeding. Technical Report. Stanford University.
4. Marcel Boehme. 2014. Automated Regression Testing and Verification of Complex Code Changes. Ph.D. Dissertation. National University of Singapore.
5. Human-In-The-Loop Automatic Program Repair
Cited by
11 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献