2023 · 108 citations · 38 references
ComputingComputer ScienceCybersecurityComputer SecurityMachine LearningVulnerability AnalysisSoftware Security EngineeringSoftware Vulnerability DiscoveryStatic Vulnerability AnalysisDeep Learning For Vulnerability DetectionModel InterpretationModel Performance AnalysisModel Performance EvaluationModel Robustness AnalysisModel Robustness AssessmentRobustnessVulnerability Detection
Deep learning (DL) models of code have recently reported great progress for vulnerability detection. In some cases, DL-based models have outperformed static analysis tools. Although many great models have been proposed, we do not yet have a good understanding of these models. This limits the further advancement of model robustness, debugging, and deployment for the vulnerability detection. In this paper, we surveyed and reproduced 9 state-of-the-art (SOTA) deep learning models on 2 widely used vulnerability detection datasets: Devign and MSR. We investigated 6 research questions in three areas, namely model capabilities, training data, and model interpretation. We experimentally demonstrated the variability between different runs of a model and the low agreement among different models' outputs. We investigated models trained for specific types of vulnerabilities compared to a model that is trained on all the vulnerabilities at once. We explored the types of programs DL may consider “hard” to handle. We investigated the relations of training data sizes and training data composition with model performance. Finally, we studied model interpretations and analyzed important features that the models used to make predictions. We believe that our findings can help better understand model results, provide guidance on preparing training data, and improve the robustness of the models. All of our datasets, code, and results are available at https://doi.org/10.6084/m9.figshare.20791240.
38
Marco Túlio Ribeiro, Sameer Singh, Carlos Guestrin · 2016 · 15.2K citations
Thomas J. McCabe · IEEE Transactions on Software Engineering · 1976 · 5.8K citations
THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.<sup>1</sup>
Paul Jaccard · New Phytologist · 1912 · 4.5K citations · Full text
Life Sciences, Botany, Ecology +10
Axiomatic Attribution for Deep Networks
Mukund Sundararajan, Ankur Taly, Qiqi Yan · arXiv (Cornell University) · 2017 · 2.6K citations · Full text
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Zhangyin Feng, Daya Guo, Duyu Tang et al. · 2020 · 2.4K citations · Full text