Wolves or Sheep: How Xray Avoids False Positives in Vulnerabilities Scans

You probably know the story of “the boy who cried ‘Wolf!’” In the ancient fable, villagers tire of a shepherd’s false alarms, and stop paying attention to them.

That’s a lesson for software security teams, not just schoolchildren. Raising concerns about threats that turn out to be flimsy or false erodes the trust that binds your department, and even the faith your customers have in you.

It’s also vital for the tools that drive your DevSecOps and continuous security efforts. Knowing the likely risks of your application’s open source components is essential, but you need to trust that the issues reported by your automated vulnerability scanning represent real problems, not phantom ones.

Positive Security

At JFrog, our aim is to help speed your releases to production. So anything that can drain your team of scarce time has our attention. False positives — reports of problems that don’t exist — are a burden for developers and security teams. Every report of a vulnerability demands time from an engineer to review and investigate. A false positive means scarce time is misspent, and they can add up to wasted hours.

Why do false positives happen? Sometimes a software component is incorrectly identified. Other times, a scanning algorithm might be too coarse to recognize a component’s corrected version. Or the database of vulnerabilities may be inaccurate, deprecated, or stale.

Even when scanning correctly identifies use of a vulnerable package, does it really deserve your attention? Flaws are plentiful; you may wish to be alerted only to those that are genuinely high-risk, such as the recently disclosed SpringShell vulnerability. Context matters too: a vulnerable function isn’t a threat if it’s never used, or if it’s mitigated by configuration settings.

That’s why JFrog and our top-notch Security Research Team work to continuously improve Xray’s analysis and reporting. Here are some of the ways that Xray helps you focus your scarce time on what matters.

Enriched Threat Data

The industry’s foundation of vulnerability data is the National Vulnerabilities Database of Common Vulnerabilities and Exposures (CVEs), a catalog of publicly disclosed security advisories issued by vendors and researchers.

Did You Know?

JFrog is authorized as a CVE Numbering Authority, and collaborates with the global security community to accelerate threat detection. Since its formation, the JFrog Security Research Team has identified and verified over 400 zero-day vulnerabilities and over 300 open-source malicious package threats in  open source repositories.

Xray supplements its CVE data with additional metrics and information from JFrog security research experts. When Xray reports detected CVEs, those with JFrog-enriched data are identified with the research team icon ().

Enhanced Scoring – JFrog Research Severity

The vulnerability metrics of the Common Vulnerability Scoring System (CVSS) help identify the severity of threat of each CVE. However, in many cases this score computed from fixed weights can be an insufficient method for understanding the actual risk from a vulnerability.

Xray supplements much of its CVE data with an additional JFrog Security Severity ranking. The rank determined by JFrog’s proprietary metric is the result of a deep analysis by our Security Research Team of the chances of real-world exploitation.

The JFrog Security Severity provides this enriched threat ranking for the vulnerability that additionally considers:

• The number of possible ways to exploit the vulnerability
• The difficulty of implementing an exploit
• The number of documented real-world attacks
• The potential impact when exploited

Every JFrog Security Severity score is accompanied by the research team’s list of reasons why the score was assigned, so that you have full insight into our assessment.

Enhanced Details

Every CVE in the public database includes some details on the vulnerability and the recommended way to remediate (for example, to use a newer version of the package). This information is available in Xray with every threat report.

JFrog customers also get access to enriched details for many CVEs that provide a deeper technical overview. This empowers security teams to better understand each vulnerability’s risk and prioritize its remediation. 

These details from JFrog security research include helpful information such as identifying the specific prerequisite conditions that enable exploitation and offering detailed technical solutions for mitigation.

Contextual Analysis

A mistake isn’t always a mistake. Sometimes it’s only an opportunity.

In the same way, a vulnerable package in your application doesn’t always mean the software you deliver is vulnerable to an exploit. Your app may never use the function that causes the problem; you may have incorporated a patch; your compilation flags may have neutralized it; the vulnerability may not be relevant to your configuration.

Context matters. That’s why Xray scans your binaries and images intelligently, taking context into account. Xray goes well beyond any simple match of dependencies against the list of CVEs, and performs a holistic analysis that examines the environment to determine whether a CVE applies.

Xray’s unique contextual threat analysis helps you pinpoint your most critical security gaps, so that you can prioritize accordingly and address them right away. Equally important, it helps screen out the noise most other software component analysis tools produce, keeping security teams focused on real threats, not rabbit holes.

Remediation Advice

Once you’ve identified an issue that’s a true threat, JFrog provides advice on how to remediate the vulnerable dependency with a new version (if one is available), or ways to mitigate the vulnerability with a code or configuration fix.

Be a Smart Shepherd

Out of nearly 174,000 CVEs (over 20,000 reported in 2021 alone), nearly one-third score as high or critical. But as we’ve seen, not every vulnerability is necessarily a real threat.

The time-saving benefits of Xray’s deep analysis and enriched threat data are most certainly real. Having the greater ability through JFrog’s security research to screen, evaluate, and mitigate threats appropriately to their likelihood and context frees up your team’s time to focus on those vulnerabilities that are a true danger.

JFrog’s deep scanning, analysis and expertise helps you to better tend to your flock of software, while keeping the true wolves that threaten your business at bay.