Static Application Security Testing (SAST) Tools for Smart Contracts: How Far Are We?

Author:

Li Kaixuan1ORCID,Xue Yue2ORCID,Chen Sen3ORCID,Liu Han1ORCID,Sun Kairan4ORCID,Hu Ming4ORCID,Wang Haijun5ORCID,Liu Yang4ORCID,Chen Yixiang1ORCID

Affiliation:

1. East China Normal University, Shanghai, China

2. Metatrust Labs, Singapore, Singapore

3. Tianjin University, Tianjin, China

4. Nanyang Technological University, Singapore, Singapore

5. Xi'an Jiaotong University, Xi'an, China

Abstract

In recent years, the importance of smart contract security has been heightened by the increasing number of attacks against them. To address this issue, a multitude of static application security testing (SAST) tools have been proposed for detecting vulnerabilities in smart contracts. However, objectively comparing these tools to determine their effectiveness remains challenging. Existing studies often fall short due to the taxonomies and benchmarks only covering a coarse and potentially outdated set of vulnerability types, which leads to evaluations that are not entirely comprehensive and may display bias. In this paper, we fill this gap by proposing an up-to-date and fine-grained taxonomy that includes 45 unique vulnerability types for smart contracts. Taking it as a baseline, we develop an extensive benchmark that covers 40 distinct types and includes a diverse range of code characteristics, vulnerability patterns, and application scenarios. Based on them, we evaluated 8 SAST tools using this benchmark, which comprises 788 smart contract files and 10,394 vulnerabilities. Our results reveal that the existing SAST tools fail to detect around 50% of vulnerabilities in our benchmark and suffer from high false positives, with precision not surpassing 10%. We also discover that by combining the results of multiple tools, the false negative rate can be reduced effectively, at the expense of flagging 36.77 percentage points more functions. Nevertheless, many vulnerabilities, especially those beyond Access Control and Reentrancy vulnerabilities, remain undetected. We finally highlight the valuable insights from our study, hoping to provide guidance on tool development, enhancement, evaluation, and selection for developers, researchers, and practitioners.

Publisher

Association for Computing Machinery (ACM)

Reference77 articles.

1. Testing Smart Contracts

2. Detecting equality of variables in programs

3. Hyperledger fabric

4. Satisfiability Modulo Theories

5. George S Boolos, John P Burgess, and Richard C Jeffrey. 2002. Computability and logic. Cambridge university press.

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3