Affiliation:
1. College of Intelligence and Computing, Tianjin University, Tianjin, China
Abstract
Compilers are a kind of important software, and similar to the quality assurance of other software, compiler testing is one of the most widely-used ways of guaranteeing their quality. Compiler bugs tend to occur in compiler optimizations. Detecting optimization bugs needs to consider two main factors: (1) the
optimization flags
controlling the accessability of the compiler buggy code should be turned on; and (2) the
test program
should be able to trigger the buggy code. However, existing compiler testing approaches only consider the latter to generate effective test programs, but just run them under several pre-defined optimization levels (e.g.,
-O0
,
-O1
,
-O2
,
-O3
,
-Os
in GCC).
To better understand the influence of compiler optimizations on compiler testing, we conduct the first empirical study, and find that (1) all the bugs detected under the widely-used optimization levels are also detected under the explored optimization settings (we call a combination of optimization flags turned on for compilation
an optimization setting
), while 83.54% of bugs are only detected under the latter; (2) there exist both inhibition effect and promotion effect among optimization flags for compiler testing, indicating the necessity and challenges of considering the factor of compiler optimizations in compiler testing.
We then propose the first approach, called
COTest
, by considering both factors to test compilers. Specifically, COTest first adopts machine-learning (the XGBoost algorithm) to model the relationship between test programs and optimization settings, to predict the bug-triggering probability of a test program under an optimization setting. Then, it designs a diversity augmentation strategy to select a set of diverse candidate optimization settings for prediction for a test program. Finally, Top-K optimization settings are selected for compiler testing according to the predicted bug-triggering probabilities. Then, it designs a diversity augmentation strategy to select a set of diverse candidate optimization settings for prediction for a test program. Finally, Top-K optimization settings are selected for compiler testing according to the predicted bug-triggering probabilities. The experiments on GCC and LLVM demonstrate its effectiveness, especially COTest detects 17 previously unknown bugs, 11 of which have been fixed or confirmed by developers.
Funder
National Natural Science Foundation of China
Publisher
Association for Computing Machinery (ACM)
Reference68 articles.
1. 2021. GCC. Retrieved December 1st 2021 from https://gcc.gnu.org/.
2. 2021. IBM XL C Compiler. Retrieved December 1st 2021 from https://www.ibm.com/products/xl-cpp-linux-compiler-power.
3. 2021. Intel C++ Compiler Classic. Retrieved December 1st 2021 from https://software.intel.com/content/www/us/en/develop/articles/oneapi-c-compiler-release-notes.html.
4. 2021. LLVM. Retrieved December 1st 2021 from https://llvm.org/.
5. 2021. OpenJ9 JIT Compiler. Retrieved December 1st 2021 from https://www.eclipse.org/openj9/docs/jit/.
Cited by
11 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Statfier: Automated Testing of Static Analyzers via Semantic-Preserving Program Transformations;Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering;2023-11-30
2. FLUX: Finding Bugs with LLVM IR Based Unit Test Crossovers;2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE);2023-09-11
3. Vectorizing Program Ingredients for Better JVM Testing;Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis;2023-07-12
4. Silent Compiler Bug De-duplication via Three-Dimensional Analysis;Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis;2023-07-12
5. Achieving Last-Mile Functional Coverage in Testing Chip Design Software Implementations;2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP);2023-05