Abstract
AbstractCompiler fuzzing techniques require a means of generating programs that are free from undefined behaviour (UB) to reliably reveal miscompilation bugs. Existing program generators such as Csmith achieve UB-freedom by heavily restricting the form of generated programs. The idiomatic nature of the resulting programs risks limiting the test coverage they can offer, and thus the compiler bugs they can discover. We investigate the idea of adapting existing fuzzers to be less restrictive concerning UB, in the practical setting of C compiler testing via a new tool, CsmithEdge, which extends Csmith. CsmithEdge probabilistically weakens the constraints used to enforce UB-freedom, thus generated programs are no longer guaranteed to be UB-free. It then employs several off-the-shelf UB detection tools and a novel dynamic analysis to (a) detect cases where the generated program exhibits UB and (b) determine where Csmith has been too conservative in its use of safe math wrappers that guarantee UB-freedom for arithmetic operations, removing the use of redundant ones. The resulting UB-free programs can be used to test for miscompilation bugs via differential testing. The non-UB-free programs can still be used to check that the compiler under test does not crash or hang. Our experiments on recent versions of GCC, LLVM and the Microsoft Visual Studio Compiler show that CsmithEdge was able to discover 7 previously unknown miscompilation bugs (5 already fixed in response to our reports) that could not be found via intensive testing using Csmith, and 2 compiler-hang bugs that were fixed independently shortly before we considered reporting them.
Funder
Engineering and Physical Sciences Research Council
Publisher
Springer Science and Business Media LLC
Reference56 articles.
1. Address Sanitizer (2012) https://clang.llvm.org/docs/AddressSanitizer.html
2. Babokin D (2019) Comment on running one million Yarpgen programs https://twitter.com/DmitryBabokin/status/1134907976085516290 [Online; accessed 24-February-2022]
3. Barr ET, Harman M, McMinn P, Shahbaz M, Yoo S (2015) The oracle problem in software testing: A survey. IEEE Transactions on Software Engineering (TSE) 41(5)
4. Bugzilla GCC (2020a) Bug 93744 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93744 [Online; accessed 24-February-2022]
5. Bugzilla GCC (2020b) Bug 94809 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94809 [Online; accessed 24-February-2022]
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. API-Driven Program Synthesis for Testing Static Typing Implementations;Proceedings of the ACM on Programming Languages;2024-01-05
2. Beyond the Coverage Plateau: A Comprehensive Study of Fuzz Blockers (Registered Report);Proceedings of the 2nd International Fuzzing Workshop;2023-07-17
3. Fuzzing Loop Optimizations in Compilers for C++ and Data-Parallel Languages;Proceedings of the ACM on Programming Languages;2023-06-06
4. Program Reconditioning: Avoiding Undefined Behaviour When Finding and Reducing Compiler Bugs;Proceedings of the ACM on Programming Languages;2023-06-06
5. Where Did My Variable Go? Poking Holes in Incomplete Debug Information;Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2;2023-01-27