Constructing exception handling chains for testing Java virtual machine implementations

Abstract

AbstractThe Java virtual machine (JVM) is the cornerstone of the Java platforms. A JVM's exception handling implementation interrupts, when the objective application encounters an exception (or an error), the normal execution of the application and performs specific handling tasks. However, little research has been done in systematically validating JVMs' exception handling implementations—test programs or even applications need to be carefully designed for throwing/catching exceptions at runtime; a JVM's exception handling implementation is also complicated, making it challenging to design tests for testing all of its functionalities. Inspired by the recent success of fuzz testing of compilers and JVM implementations, we introduce EHCBuilder, the first technique for fuzzing JVMs' exception handling implementations. The key idea is to construct exception handling chains, each of which abstracts a program's execution into a sequence of exception throwings, catchings, and/or handlings. A classfile seed can then be mutated into test programs with diverse exception handling chains, enabling (1) exceptions to be continuously thrown and caught at runtime, and (2) JVMs' exception handling implementations to be much more thoroughly tested. We have implemented EHCBuilder and evaluated EHCBuilder on popular JVM implementations including OpenJDK's HotSpot, Eclipse's OpenJ9, Azul's Zulu, and Oracle's GraalVM. Our results show that EHCBuilder can generate programs with very intricate exception handling chains and reveal differences among JVMs' exception handling implementations: Up to thousands of lines of source code in HotSpot's exception handling implementation are covered more than the original benchmarks; during 39 K iterations, EHCBuilder generates exception handling chains of different lengths, revealing 258 runtime differences. We classify the differences into four categories, and reveal a fast throw issue confirmed by HotSpot developers and another initCause issue confirmed by the OpenJ9 community.