Abstract
Coverage analysis is increasingly important for enhancing deep learning (DL) framework testing. However, current code coverage analyzers face challenges with low compatibility and high overhead when applied to DL frameworks. Our findings indicate that state-of-the-art Python analyzers, such as coverage.py and SlipCover, introduce an overhead of at least 1 \(\times\) . In the context of C coverage, the widely-used analyzer gcc is compatible only with TensorFlow and imposes a substantial overhead of 130,166$\times$. These issues with compatibility and overhead significantly limit the utility of coverage analysis in many applications, including testing and fuzzing, and prevent its adoption in deployment scenarios.In this paper, we propose DCOV, a lightweight coverage analyzer designed explicitly for DL frameworks to address the above challenges. DCOV leverages run-time instrumentation for Python code and compile-time instrumentation for C code. Additionally, we decouple instrumentation and coverage analysis functions by an interface-based instrumentation design, allowing Python and C probes to share the same coverage analysis algorithm. Furthermore, DCOV minimizes execution and analysis overheads through several strategies, including coverage granularity optimization, parallel processing and bit counting algorithm optimization.Our empirical evaluation demonstrates DCOV's applicability to popular deep learning frameworks, such as TensorFlow and PyTorch, with a minimal overhead of approximately 18 milliseconds per single run. In addition, by adopting parallel processing and bit counting algorithm optimization, DCOV reduces analysis overhead by 99%. Moreover, by integrating DCOV with existing DL framework testing systems, we observe a 2.8$\times$ speedup and uncover two previously unknown bugs in TensorFlow (one of them being accepted).