The CVSS Fallacy – can you trust the world’s most popular vulnerability metric?
The NVD defines one of the usages of CVSS as “a factor in prioritization of vulnerability remediation” and it is the current de-facto vulnerability metric, often seen as infallible guidance and a crucial element in many compliance processes. In our session we will go over real-world CVE examples, demonstrating cases and entire categories where CVSSv3.1 falls short of providing an accurate assessment, both due to its design and its various mishandlings. The session will also touch upon specific indicators in the CVE description that can raise the confidence in a CVSS score, and vice versa.
Hear from expert Brian Moussalli, Security Research Tech Lead at JFrog, who has over 13 years of experience in cyber security, experienced in security research, reverse engineering and malware analysis. He specializes in vulnerability analysis, threat intelligence research and automated threat detection.
Transcription:
Brian Moussalli:
Hello, everyone. Today we’re going to talk about the CVSS fallacy and ask, can you trust the world’s most popular vulnerability metric?
My name is Brian Moussalli. I’m a security research tech lead at at JFrog Security. We’re going to talk about what is the common vulnerability scoring system, why isn’t CVSS an accurate metric, and we’ll take a look at a couple of real life examples.
What is the common vulnerability scoring system? It is a free and open standard for assessing the severity of software and hardware vulnerabilities, developed by the Forum of Incident Response and Security teams. Its current version is 3.1, which was released in 2019. The CVSS actually offers three rating methods.
The first one is base score, which reflects the severity of a vulnerability according to several characteristics, which we will discuss in detail in the next slide. The second one is a temporal score, which adjusts the base severity of a vulnerability based on factors that change over time, such as availability of exploit code. The third score is the environmental score, which adjusts both base and temporal severities to a specific computing environment according to the presence or lack thereof of mitigations in that environment.
What is the CVSS used for? Complex environments are full of third-party software packages, hardware, et cetera. Whether it’s your organizational network or the docker containers you use in your production environment, almost everything is composed of third-party software or hardware components, which can turn out to be vulnerable at any time. This fact poses a risk on your organization or your product and requires remediation measures to be taken, but remediation in production environments may be costly in terms of time and work. For this reason, we would like to prioritize vulnerability remediation in product environments.
Enter CVSS. CVSS gives us a score, which helps us understand how severe a threat is and how much we should panic. Nobody likes being exposed to a hacking attempt, and since remediation requires the work of your software developers or your IT or DevOps teams, it should be done correctly.
How is the base score calculated? We have several metrics from which the exploitability score is composed and another set of metrics for the impact score. These two combine give the base score and the vector string, which we can see at the picture below.
The access vector metric shows how this vulnerability may be exploited, if it’s a local attack or one that can be carried out using a network connection. The access complexity metric refers to the difficulty of exploiting the vulnerability. The privileges required metrics refers to the privileges that an attacker must have in order to exploit the vulnerability. The user interaction metric specifies whether a human interaction other than the attackers is required to exploit this vulnerability. The scope metric refers to the impact of this vulnerability on other modules or resources. The confidentiality metric refers to the impact on the confidentiality of information resources managed by the vulnerable component, high being complete loss of confidentiality. The integrity metric refers to the impact on the components protected resource. For example, if an attacker is able to modify files after exploitation, high being complete loss of integrity. The availability metric refers to the impact on the components availability, high being complete, loss of availability like in denial of service.
Why isn’t CVSS an accurate metric? Crucial components that the CVSS fails to account for are code prerequisites, like what are the vulnerable functions or are all uses of these functions vulnerable? This information can be expressed in the current base or vector string configuration prerequisites, like what is the vulnerable configuration of the service or is it the default configuration or is it a common configuration. Environment prerequisites are like are all Windows or Linux or Macs, are they all vulnerable?
Let’s take a look at some examples to see what I mean exactly. CVE-2016-10749. It’s a buffer overflow in cJSON. Let’s take a look at the CVE’s description. The first function, the parse string function in cJSON before October 2nd, 2016, has a buffer over-read as demonstrated by a string that begins with a double quote character and ends with a backslash. The base score for this vulnerability is 9.8, which is critical. Is that so?
Let’s take a look. As we can see in the exploit example, in order to exploit this vulnerability, an attacker must control the input of the cJSON parse function. This means that the exploitability and impact of the CVE are highly contextual. It depends on how the vulnerable function is used in our product or environment. If the cJSON library is installed but is installed but nobody uses it, it’s really, really not interesting. On the other hand, it’s very interesting if a piece of software uses the cJSON parse function with user-controlled input. The attack vector in the vector string reads network but this is actually not so accurate.
The CVSS base score doesn’t take into account whether the attack vector is context dependent, meaning it requires the vulnerable software component to call this function with user input. User input in this case is the key to understanding whether the CVE is relevant for me and should cause the panic a critical CVE should cause. The same goes for the privileges required metrics, which says none in this case, but the privileges that are required to exploit this vulnerability are dependent on the context in which the library is used. It may only be exploitable in a system then that requires prior authentication. The same goes for the impact score. Impact may also be context dependent and rely on the permissions in which a vulnerable process or service runs.
Let’s take a look at another CVE. The next one is an out-of-bounds Write vulnerability in Apache’s mod_sed module. It’s a filter model which offers the same capabilities as new stream editor command line tool, but for HTTP requests and responses. Again, we see it received a score of 9.8, which is critical, but is it really?
Let’s see. mod_sed is not used by default, and actually it’s quite rare. It’s a rare filter module to be used. In order for Apache to be exploitable, it must be configured to use the sed module. As we can see in this configuration, this mod_sed module may be defined as an input filter for a certain endpoint. In this case, it’s the HTTP route directory, meaning all requests to the server will be handled by this module, but this is extraordinary. In reality, the correct vulnerable endpoint must be guessed by an attacker, or they should have access to Apache’s configuration file, which is very, very, very unlikely. I suggest that the attack complexity shouldn’t be low in this case.
The CVSS score doesn’t take into consideration other mitigations such as limiting the request size, which can help in the case of this CVE. Another requirement for successful exploitation of the CVE is the size of the data that’s being sent to the server. It requires over two gigabytes of data to be sent to the server. This is another thing that isn’t expressed in the CVSS, but is crucial in terms of mitigation and remediation. This vulnerability can be mitigated by using the LimitRequestBodySize directive in Apache’s configuration, which is extremely easy to add. In my opinion, this really, really reduces the risks from a critical 9.8 to something much lower.
Conclusions: CVSS is useful for initial evaluation, but fails to account for several prerequisites like code prerequisites. When a specific function is vulnerable, the attack vector value should be contextual in my opinion. Contextual option doesn’t exist so everybody just uses network, which is very inaccurate and causes confusion, or maybe not confusion but causes CVEs to get really high base scores, exploitation scores. The same goes for impact scores. Impact score scores are very much contextual. In many times, they rely on the specific impact that a vulnerability may have. If it’s the privileges that the process has or service has, configuration prerequisites are also not being taken into account or how common is the vulnerable configuration or is it the default configuration? We can specify all that information in the CVSS base score and in the vector string.
Also, environment perquisites like Windows or Linux or macOS. Are they all vulnerable? There are certain libraries or software services that can run on all three of them or other operating systems, but are they all vulnerable just the same? We can’t really tell by the base score or the exploitability score or the impact score. The key thing is context, context, context. Understanding the full impact and severity of a vulnerability requires full context. This is what we need in order to best utilize the CVSS score in our organization when we prioritize what needs to be remediated and what needs to be fixed.
Here’s an example from JFrog’s Xray in which you can see contextual analysis. You can see that other than the CVSS score shown to the user, there’s also a JFrog severity metric and the contextual analysis field, which tell the user whether the CVE is actually applicable in their case. Below, you can see that this depends on the client’s first-party code actually using the vulnerable functions in a way that poses a real threat to their environment. The client is given specific pointers to the vulnerable code with the exact file and line. This helps them remediate this vulnerability in the best possible way, whether it’s changing the parameters that are sent to the function, which make it vulnerable, or making sure that not like that the user input doesn’t find its way into the vulnerable function in case user input may be a problem.
Bottom line is we should always consider the full context of the CVE and unfortunately, CVSS is not enough to understand and help us prioritize how we should direct our resources in remediation. If you have any questions, feel free to email me at brianm@jfrog.com. Thank you very much.
