Comparing Vulnerability Severity and Exploits Using Case-Control Studies

TLDR

Rule‑based policies for mitigating software risk recommend using the CVSS score to gauge vulnerability risk. The study aims to determine whether the CVSS score reflects real‑world exploitation risk and how it might be improved. The authors employ a case‑control study, analogous to the lung cancer–smoking paradigm, to quantify the relationship between risk factors (e.g., CVSS score, proof‑of‑concept exploits, black‑market exploits) and exploitation outcomes using publicly available vulnerability data. The analysis shows that high CVSS scores alone do not guide effective patching, proof‑of‑concept exploits are a stronger predictor of exploitation, and responding to black‑market exploit presence achieves the greatest risk reduction. The study is conducted in the U.S.

Abstract

(U.S.) Rule-based policies for mitigating software risk suggest using the CVSS score to measure the risk of an individual vulnerability and act accordingly. A key issue is whether the ‘danger’ score does actually match the risk of exploitation in the wild, and if and how such a score could be improved. To address this question, we propose using a case-control study methodology similar to the procedure used to link lung cancer and smoking in the 1950s. A case-control study allows the researcher to draw conclusions on the relation between some risk factor (e.g., smoking) and an effect (e.g., cancer) by looking backward at the cases (e.g., patients) and comparing them with controls (e.g., randomly selected patients with similar characteristics). The methodology allows us to quantify the risk reduction achievable by acting on the risk factor. We illustrate the methodology by using publicly available data on vulnerabilities, exploits, and exploits in the wild to (1) evaluate the performances of the current risk factor in the industry, the CVSS base score; (2) determine whether it can be improved by considering additional factors such the existence of a proof-of-concept exploit, or of an exploit in the black markets. Our analysis reveals that (a) fixing a vulnerability just because it was assigned a high CVSS score is equivalent to randomly picking vulnerabilities to fix; (b) the existence of proof-of-concept exploits is a significantly better risk factor; (c) fixing in response to exploit presence in black markets yields the largest risk reduction.