Affiliation:
1. Shenzhen University, Shenzhen, China
2. State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, China
3. Shenzhen University & Shenzhen Technology University, Shenzhen, China
Abstract
As a new programming paradigm, deep neural networks (DNNs) have been increasingly deployed in practice, but the lack of robustness hinders their applications in safety-critical domains. While there are techniques for verifying DNNs with formal guarantees, they are limited in scalability and accuracy. In this paper, we present a novel counterexample-guided abstraction refinement (CEGAR) approach for scalable and exact verification of DNNs. Specifically, we propose a novel abstraction to break down the size of DNNs by over-approximation. The result of verifying the abstract DNN is conclusive if no spurious counterexample is reported. To eliminate each spurious counterexample introduced by abstraction, we propose a novel counterexample-guided refinement that refines the abstract DNN to exclude the spurious counterexample while still over-approximating the original one, leading to a sound, complete yet efficient CEGAR approach. Our approach is orthogonal to and can be integrated with many existing verification techniques. For demonstration, we implement our approach using two promising tools
Marabou
and
Planet
as the underlying verification engines, and evaluate on widely-used benchmarks for three datasets
ACAS
Xu
,
MNIST
and
CIFAR-10
. The results show that our approach can boost their performance by solving more problems in the same time limit, reducing on average 13.4%–86.3% verification time of
Marabou
on almost all the verification tasks, and reducing on average 8.3%–78.0% verification time of
Planet
on all the verification tasks. Compared to the most relevant CEGAR-based approach, our approach is 11.6–26.6 times faster.
Publisher
Association for Computing Machinery (ACM)
Reference56 articles.
1. 2021. 2nd International Verification of Neural Networks Competition (VNN-COMP’21). Retrieved 2022 from https://sites.google.com/view/vnn2021
2. DeepAbstract: Neural Network Abstraction for Accelerating Verification
3. Rudy Bunel Ilker Turkaslan Philip H. S. Torr Pushmeet Kohli and M. Pawan Kumar. 2017. Piecewise Linear Neural Network verification: A comparative study. CoRR abs/1711.00455(2017). arxiv:1711.00455 http://arxiv.org/abs/1711.00455
4. Towards Evaluating the Robustness of Neural Networks
5. Counterexample-Guided Abstraction Refinement