Xray Quick Start Guide

Patrick Russell
2019-08-08 23:08

Xray installation quick start guide

This guide is intended to get Xray 2.X up and scanning binaries as quickly as possible. It has a basic troubleshooting section at the end for the most commonly seen errors.

0. Before getting started

Required Hardware

Xray requires the following minimum hardware:

– Storage: At least 100 GB 

– CPU Processors: 8 cores

– RAM: 16 GB

The storage requirement is very important:

1. To scan packages, Xray downloads the entire artifact to its local disk
2. Xray's vulnerability database requires about 16 GB and will grow over time

A recommended setup should have a 100+ GB disk mounted to the root of the file system. For the RPM and Debian installations, Xray's system files will be spread across the /var and /opt directories.

(Optional) External Databases

Xray needs two databases to store its scan history and vulnerabilities. Xray's installer can set this up automatically. If external databases are required, install them before installing Xray.

Xray requires these exact versions of Mongo and Postgres:

– PostgreSQL 9.6.11
– MongoDB 3.2.6

1. Install Xray

Xray's installers will automatically install the 4 Xray microservices, and the 3 external services:

[External services]
MongoDB – The artifact license / vulnerability database
PostgreSQL – The scan history database
RabbitMQ – A message server to coordinate scans

[Xray services]
Server – The web server frontend
Indexer – Decompresses binary files
Analysis – Checks decompressed binaries for license / security violations
Persist – Saves the Analysis results to the Postgres database

Download Xray's installer

Use the official JFrog Xray downloads site to pull down the right installer file. 

The Centos, Ubuntu, Debian, and RedHat installers are full binary files, and can be installed in an offline air-gapped environment. The Docker installer is an SH script that will pull the images from docker.bintray.io.
Run the installer

[Docker installation]
./xray install

[RPM installation – RedHat – CentOS]
./installXray-redhat.sh

[DEB installation – Ubuntu – Debian]
./installXray-ubuntu.sh

[Example installation answers – Options mentioned are the defaults]

2019-07-24 21:35:11  [227 installXray-ubuntu.sh] Starting Xray installation
Verifying Xray prerequisites …
[…]
2019-07-24 21:35:16  [489 installXray-ubuntu.sh] Xray data must be set on Xray first installation
Provide Xray data folder [/var/opt/jfrog/xray/data]: /var/opt/jfrog/xray/data
[…]
Are you adding this node to an existing cluster? (not relevant for the first cluster node) [Y/n]: n
Would you like to install PostgreSQL instance? [Y/n]: y
Would you like to install MongoDB instance? [Y/n]: y
[…]
2019-07-24 21:35:28  [816 installXray-ubuntu.sh] Postgres home must be set on Xray first installation
Provide Postgres home folder [/var/opt/jfrog/postgres]: /var/opt/jfrog/postgres

2. Get Xray Ready

When you reach the Xray web page for the first time, there is an Onboarding Wizard to get things started. The main purpose of the wizard is to set up the Xray Artifactory connection.

Connect Xray to Artifactory

Xray uses a 2-way connection to communicate with Artifactory. Xray needs to be able to reach Artifactory, and Artifactory needs to reach Xray. 

Xray needs to know 2 things to set up this link:
1. How Artifactory can reach Xray. This is the "Xray Base URL":

2. How Xray can reach Artifactory. The "Artifactory URL" configured after setting up the Xray Base URL:

For the "Artifactory Admin User" credentials, be sure to use a system account with a password that will not change. It is difficult to update this password as of Xray 2.X.

After this, the onboarding wizard will ask which repositories to add to Artifactory:

Add the repositories you need to, but leave "index existing artifacts" unchecked for now. You need to set up Xray to scan the artifacts correctly the first time.

Sync the Xray Database

Once Xray is connected to Artifactory, start the database synchronization. Xray is most effective at finding vulnerabilities and licenses when its vulnerability and license databases are fully synced. 

The Online Sync requires a connection to these websites:

1. https://dl.bintray.com
2. https://akamai.bintray.com
3. https://jxray.jfrog.io

3. Scan Artifacts

Add the first Policy and Watch

Xray is almost ready to work. However, before indexing artifacts, the system needs to know what to do with the scan results. Without defining a Policy or a Watch, Xray's will not log security/license violations.

Policies determine what actions Xray takes after scanning a file.

Watches determine what resources a Policy applies to.

Every organization will have different needs, but the best thing to get started is to have a broad Watch with one "logging" Policy. This way, you can observe all of the violations found on Artifactory before taking actions such as blocking downloads.

1. Create a Policy that only logs a security vulnerability:

2. Create a Watch with this Policy, and use "All Artifacts":

 

Scan Artifactory's artifacts

Xray is ready to go! It's installed, connected to Artifactory, has an up to date database, and knows what to do with its scan results. 

Time to give Xray some data to scan. Do this by going to Admin -> Configuration -> Artifactory and select "Configure Indexed Repositories". Add the repositories to scan, and then click "Index Existing":

Xray will scan the artifacts, and the Watch will have violations to investigate further. 

The Xray Artifactory system will now automatically scan any files that are uploaded to the watched repositories. The "Index Existing" button scans everything that was in the repository before it was added to Xray.

Xray has been set up. Some good next steps would be:

1. Review the scan results to build a picture of what in-use packages have violations

2. Set up watches that watch specific repositories, and take specific action such as blocking downloads

3. Set up a CI Build to take advantage of the Build Scan feature

 

Troubleshooting

Troubleshooting the installer

Xray's installer might encounter an error. If it does, the first thing to try is to remove the packages already installed and try again.

The steps to reset the Xray environment (Uninstalling Xray) can be found on the wiki.

Review the logs to see if another package or configuration is interfering with the installation. The installer logs failures in the "installXray-<OS>.<Timestamp>.log" file.

If this still doesn't help, get in touch with JFrog Support with the log file for further assistance. 

Troubleshooting the Artifactory <> Xray connection

The URLs should be reachable from each host. Test this using these curl commands:

[Xray host -> Artifactory URL – "Xray cannot connect to Artifactory"]
curl http://artifactory.com:8081/artifactory/api/system/ping
[Expected response]
OK

Update the Artifactory URL with the URL that works.

[Artifactory host -> Xray URL – "Xray can reach Artifactory but Artifactory cannot reach Xray"]
curl http://xray.com:8000/api/v1/system/ping
[Expected response]
{"status":"pong"}

Update the Xray Base URL with the URL that works.

Troubleshooting Scanning problems

Xray uses an Access Token to download files from Artifactory. To refresh the token, Xray uses the Artifactory Admin credentials created in step 2. If these credentials have changed, there will be 401 errors in the Artifactory logs.

If there is no scan history, you can delete the Artifactory connection in Xray to reset the URL or credentials. If there is a scan history that needs to be preserved, get in touch with JFrog Support.