A generic instrumentation framework for collecting dynamic information

Abstract

Performing empirical research in software testing involves executing a set of subjects against one or more test suites and measuring some characteristics of these executions. Such measures are often collected using ad-hoc instrumentation, by inserting probes in the code that collect and report dynamic information at run-time. Another possible approach is to collect the needed information by leveraging capabilities of the runtime system. Both these approaches usually result in measurement tools that are not flexible and are, thus, hard to reuse and modify. To address this problem, we present a generic framework for collecting information on the runtime behavior of a Java program. The framework allows for easily collecting different kinds of dynamic information for a set of executions of the program, such as coverage and profiling of various code entities and program traces at different levels. The framework also lets users easily define how to process the collected information. In the paper, we also present a case study that we performed to evaluate the framework, and that shows its effectiveness and efficiency.