A Systematic Evaluation Framework for Smart Contract Security Analyzers: Methods, Metrics, and Framework

Abstract

Smart contracts automate agreements in blockchain systems but their immutable nature makes them vulnerable to permanent flaws once deployed. This thesis evaluates 256 smart contract vulnerability detection tools developed between 2018 and 2024, including approaches such as fuzzing, symbolic execution, formal verification, and artificial intelligence–based analysis. Tools were classified by detection strategy (static, dynamic, hybrid), domain (academic or industry), and scope. The evaluation involved a theoretical review of architecture, usability, and documentation, alongside an empirical assessment of accuracy, speed, and false positive rates. Findings show that while certain tools excel in specific areas, none achieve balanced performance or comprehensive coverage. To address these gaps, a modular six-layer evaluation framework is introduced, defining functional areas such as code analysis, coverage, integration, and user experience. The framework offers a benchmark for tool assessment and future development. Additionally, a graph-based detection model is proposed, demonstrating improved accuracy in both binary and multi-class settings.