Xray’s “History Scan” – How to Update an Xray Watch

Patrick Russell
2019-09-05 22:03

Xray's "History Scan" – How to update an Xray Watch

Say you've made a change to a watch in Xray, like turning off a "Block downloads" policy. Yet despite making the change, nothing gets updated and the artifacts are still blocked. What's going on?

This behavior is due to how Xray scans and tracks artifacts in Artifactory, and it can be corrected by re-scanning the same artifacts to apply updated policies.

If your build is blocked or you otherwise need to un-block artifacts quickly, use the Artifactory Admin -> JFrog Xray menu's "Allow downloads of blocked artifacts" checkbox:

The proper way to update a watch, and why a re-scan is necessary, are explained in detail below.

Applying Policies on Scanned Artifacts

When Xray scans an artifact for the first time, it recursively decompresses and calculates the checksums of the binary file. 

This recursive process is used to find the Components of the artifact; for example, a JAR file inside a ZIP binary is a Component that Xray can detect. At the end of the scan, Xray saves the binary file's checksums within its database. The artifact only needs to be downloaded and scanned once using this process.

On its own, this process does not detect vulnerabilities or license issues. There are many possible things to look for when scanning a binary, so a system was developed to efficiently determine what to do with scan results.

This system is based around using Watches to track specific artifacts, and Policies to apply actions on these tracked files. Watches' policies are applied during the Analysis phase of a scan, after the binary has been Indexed and Persisted to the database.

Normally this is supposed to be done once (Or after a Database Sync for new vulnerabilities), when the binary file is first scanned. But what if you need to change the results or un-block a lot of files?

Apply on Existing Content

To implement a new watch or apply a new policy, you need to trigger a History Scan. This is done by selecting the "Apply on Existing Content" button in the Watches menu of Xray. 

Triggering a History Scan causes the Analysis process of Xray to do a deep-dive in the database. Since Xray has already scanned the items in the watched repository, it needs to search the database for an artifact's checksum to see if the watch's policy applies.

This re-scan is a database and system-intensive operation. Because of this, you cannot run a history scan on watches that use the "All Artifacts" or "All Builds" resources. Doing so forces Xray to rescan an entire Artifactory, which is not efficient and should be avoided if possible.

A good practice to follow is to avoid setting blocking policies on any "All Artifacts / All Builds" watches, as it is difficult to disable this functionality. These watches should only be used to track violations using a generic "Generate Violation" policy. 

For details on a recommended initial setup of Xray, take a look at the Xray Quick Start Guide.