Xray Single Node Manual RPM Installation

JFrog Installation & Setup Documentation
Content Type
Installation & Setup
ft:sourceType
Paligo

The RPM installation bundles Xray and all its dependencies. It is provided as native RPM packages, where Xray and its dependencies must be installed separately. Use this, if you are automating installations.

Before you proceed with the installation, review the system requirements.

Xray System Requirements
Xray Node Recommendations

Use a dedicated node for Xray with no other software running to alleviate performance bottlenecks, avoid port conflicts, and avoid setting uncommon configurations.

Xray Storage Recommendations

In most cases, our recommendation is to use an SSD drive for Xray to have better performance and it is not recommended to use an NFS drive, as it is a disk I/O-intensive service, a slow NFS server can suffer from I/O bottlenecks and NFS is mostly used for storage replication.

Xray stores node specific files, such as configuration and temporary files, to the disk. These files are exclusively used by Xray and not shared with other services. Since the local storage used for Xray services are temporary, it does not require replication between the different nodes in a multi-node/HA deployment.

Xray File Handle Allocation Limit

Use the following command to determine the current file handle allocation limit.

cat /proc/sys/fs/file-max

Then, set the following parameters in your /etc/security/limits .conf file to the lower of 100,000 or the file handle allocation limit determined above.

The example shows how the relevant parameters in the /etc/security/limits .conf file are set to 100000. The actual setting for your installation may be different depending file handle allocation limit in your system.

root hard nofile 100000
root soft nofile 100000
xray hard nofile 100000
xray soft nofile 100000
postgres hard nofile 100000
postgres soft nofile 100000
Operating Systems and Platform Support

The following table lists the supported operating systems and the versions.

Product

Debian

RHEL

Ubuntu

Windows Server

Amazon Linux

Xray

10.x, 11.x

8.x, 9.x

20.04, 22.04

Operating Systems - End of Support

As part of JFrog commitment to maintain the security and reliability of the JFrog Platform, Artifactory will officially run with Node.js 20.x on all installation types from Artifactory 7.77.3.

Node.js 20.x provided with Linux Archive/Debian/RPM installations (non-containerized distributions) is not supported on the following operating systems.

Hence, these operating systems will no longer supported from Artifactory version 7.77.3.

Supported Platforms

The following table lists the supported platforms.

Product

x86-64

ARM64

Kubernetes

OpenShift

Xray

1.19+

4.13+

Installation on Kubernetes environments is through Helm Charts. Supported Helm version is Helm 3+.

ARM64 Support

From version 7.41.4, Artifactory supports installation on ARM64 architecture through Helm and Docker installations. You must set up an external database as the Artifactory database since Artifactory does not support the bundled database with the ARM64 installation. Artifactory installation pulls the ARM64 image automatically when you run the Helm or Docker installation on the ARM64 platform.

ARM64 support is also available for Xray, Pipelines (in Helm installation), and Insight. ARM64 support is not available for Distribution.

Database and Third-Party Applications in Xray

Every artifact and build indexed by Xray is broken down into multiple components. These components and the relationships between each other are represented in a checksum based components graph. Xray uses PostgreSQL to store and query this components graph.

Xray supports the following versions of PostgreSQL.

  • 15.x (from version 3.78.9)

  • 14.x

  • 13.x (from version 3.18)

  • 12.x

  • 11.x

  • 10.x

Xray supports PostgreSQL 14.x and 15.x, but currently the Xray installer only bundles the binaries for PostgreSQL 13.x.

RabbitMQ is installed as part of the Xray installation for every node. In case of HA architecture, Xray uses queue mirroring between the different RabbitMQ nodes. External RabbitMQ instances are not officially supported; the recommended method of installation is to use the bundled RabbitMQ.

Xray has multiple flows, such as scanning, impact analysis, and database sync. These flows require processing completed by the different Xray microservices. Flows contain multiple steps that are completed by the Xray services. Xray uses RabbitMQ to manage these different flows and track synchronous and asynchronous communication between the microservices.

Xray also uses Erlang and DB-Util third-party applications. These packages are bundled with all Xray installers except Linux Archive.

Xray Network Ports

Xray uses the 8082 port by default for external communication.

Xray uses the following internal ports by default for communication with JFrog Platform microservices.

Microservice

Port

Xray Server

8000

Analysis

7000

Indexer

7002

Persist

7003

Router

8082, 8046, 8047, and 8049

RabbitMQ

4369, 5671, 5672, 15672, and 25672

PostgreSQL (if you use the bundled PostgreSQL database)

5432

Observability

8036

8037

gRPC

Install/Upgrade Xray 3.70.0 to 3.74.0 with RPM

Xray installation/upgrade might fail because of the missing package libltdl. If you face the error, you can download and install the package, and proceed with the installation/upgrade of Xray.

Complete the following steps to install the product.

  1. Extract the contents of the compressed archive, and go to the extracted folder.

    tar -xvf jfrog-xray-<version>-rpm.tar.gz
cd jfrog-xray-<version>-rpm

  2. Install PostgreSQL.

    You can choose to install any compatible PostgreSQL version, or use the PostgreSQL RPM bundled with the Xray installer located under /third-party/postgresql.

    PostgreSQL is required and must be installed before continuing with the next installation steps.

    Set your PostgreSQL connection details in the Shared Configurations section of the $JFROG_HOMExray/etc/system.yaml file.

  3. Install db-util.

    You can use the bundled db-utils found under /third-party/misc/.

    db-util allows Xray to interact with the Berkley DB that contains information about RPM-based Docker images. This way, Xray can index OS packages for these images.

    # This will install db-util if db_dump is not available
hash db_dump 2>/dev/null || rpm -ivh --replacepkgs ./third-party/misc/<db-utils version>.x86_64.rpm

  4. Install RabbitMQ dependencies.

    # Note : Use rpms with el7 when installing on Centos 7 and RHEL 7, el8 with RHEL8, and el9 with RHEL9
Run the following from the extracted folder.
rpm -ivh --replacepkgs ./third-party/rabbitmq/socat-<version>.x86_64.rpm
rpm -ivh --replacepkgs ./third-party/rabbitmq/erlang-<version>.x86_64.rpm

  5. Install Xray.

    You must run as a root user.

    rpm -Uvh --replacepkgs ./xray/xray.rpm

  6. Customize the product configuration.

    1. Set the Artifactory connection details.

    2. Customize the PostgreSQL Database connection details. (optional)

    3. Set any additional configurations (for example: ports, node id) using the Xray system.yaml configuration file.

      Warning

      Verify that a large file handle limitis specified before you start Xray.

  7. Start and manage the Xray service.

    systemd OS

    systemctl start|stop xray.service

    systemv OS

    service xray start|stop|status|restart

  8. Access Xray from your browser at: http://<jfrogUrl>/ui/:port.

    Go to the Xray Security & Compliance tab in the Administration module in the UI.

  9. Check the Xray Log.

    tail -f $JFROG_HOME/xray/var/log/console.log

    Configure log rotation of the console log

    The console.log file can grow quickly since all services write to it. For more information, see configure the log rotation.Configuring Log Rotation for Tomcat

Xray Product Configuration

After installing and before running Xray, you may set the following configurations.

You can configure all your system settings using the system.yaml file located in the $JFROG_HOME/xray /var/etc folder. For more information, see Xray System YAML.

If you don't have a System YAML file in your folder, copy the template available in the folder and name it system.yaml.

For the Helm charts, the system.yaml file is managed in the chart’s values.yaml.

Artifactory Connection Details for Xray

Xray requires a working Artifactory server and a suitable license. The Xray connection to Artifactory requires the following parameters.

  • jfrogUrl

    URL to the machine where JFrog Artifactory is deployed, or the load balancer pointing to it. It is recommended to use DNS names rather than direct IPs. For example: http://jfrog.acme.com or http://10.20.30.40:8082. Note that /artifactory context is not longer required.

    Set it in the Shared Configurations section of the $JFROG_HOME/xray/var/etc/system.yamlfile.

  • join.key

    This is the "secret" key required by Artifactory for registering and authenticating the Xray server.

    You can fetch the Artifactory joinKey (join Key) from the JPD UI in the User Management | Settings | Join Key.

    Set the join.key used by your Artifactory server in the Shared Configurations section of the $JFROG_HOME/xray/var/etc/system.yaml file.

Change PostgreSQL database credentials

Xray comes bundled with a PostgreSQL database out-of-the-box, which come pre-configured with the default credentials.

To change the default credentials:

# Access PostgreSQL as the Xray user adding the optional -W flag to invoke the password prompt
$ psql -d xraydb -U xray -W
  
# Securely change the password for user "xray". Enter and then retype the password at the prompt.
\password xray
  
# Verify the update was successful by logging in with the new credentials
$ psql -d xraydb -U xray -W

Set your PostgreSQL connection details in the Shared Configurations section of the $JFROG_HOME/xray/var/etc/system.yaml file.

Change RabbitMQ database credentials

Xray comes pre-installed with RabbitMQ, by setting the Erlang cookie value as the RabbitMQ password for guest users.

RPM/Debian

  1. Set the new password in the$JFROG_HOME/app/bin/rabbitmq/rabbitmq.conffile.

    default_pass = <new password>

  2. Set your RabbitMQ password in the Shared Configurations section of the $JFROG_HOME/xray/var/etc/system.yaml file.

  3. Restart all services.

    service xray restart / systemctl restart xray.service
Configure RabbitMQ to use FQDN for clustering

By default, RabbitMQ uses the short hostnames of other nodes in the cluster for communication. However, it be can be configured to use a fully qualified domain name (FQND) host name (a long hostname).

To configure RabbitMQ to use FQDN, follow these steps.

  1. Install Xray , but do not start the services.

  2. Modify the following files according to the installer type.

    • Linux and Native Installers

      In JFROG_HOME/app/bin/xray.default:
 
   export RABBITMQ_USE_LONGNAME=true

    • Common Change in All Installers

      In system.yaml:
 
shared:
  node:
    id: <long hostname>
    name: <long hostname>
## For secondary nodes only
#  shared:
#    rabbitMq:
#      active:
#        node:
#          name: <long hostname of active node>

  3. Start RabbitMQ and the Xray services.

Third Party Log Collector

Xray enables using an external log collector such as Sumologic or Splunk.

To adjust the permissions to allow the log collection service perform read operations on the generated log files.

  1. Add the log collection service user to the relevant group if needed (the user and group that installed and started Xray).

  2. Apply the user and group permissions as needed on the $JFROG_HOME/xray/var/log directory using:

    $ chmod -R 640 $JFROG_HOME/xray/var/log

  3. Adjust the group read inheritance permissions setgid bit using:

    $ chmod -R 2755 $JFROG_HOME/xray/var/log

    This command enables the generated log files to inherit the folder's group permissions.

Third Party Applications for Xray

Ensure that you install the third party application for Xray before run the Xray service.

PostgreSQL for Xray

Using Microsoft Azure PostgreSQL

Some managed databases, such as Azure, have a different username for accessing the database than the actual one inside the database. For Azure-managed PostgreSQL, the username will be, for example xray@mycompany and the actualUsername will be xray. For more information, see Xray System YAML.

Prior to Xray version 3.30

If you install an Xray version prior to 3.30, do not use a password for PostgreSQL that has special characters.Xray may not work if you configure a password that has special characters, such as~ = # @ $ /.

PostgreSQL RPM Installation

  1. Install PostgreSQL.

    # Run the following commands from the extracted jfrog-xray-<version>-rpm directory.
# Note : Use PostgreSQL rpms with el7 when installing on RHEL 7, el8 with CentOS 7 and RHEL8, and el9 with RHEL 9
 
mkdir -p /var/opt/postgres/data

# For RHEL 7 and CentOS 87
rpm -ivh --replacepkgs ./third-party/postgresql/libicu-50.2-4.el7_7.x86_64.rpm (only AWS instance)
rpm -ivh --replacepkgs ./third-party/postgresql/postgresql13-libs-13.10-1PGDG.rhel7.x86_64.rpm
rpm -ivh --replacepkgs ./third-party/postgresql/postgresql13-13.10-1PGDG.rhel7.x86_64.rpm
rpm -ivh --replacepkgs ./third-party/postgresql/postgresql13-server-13.10-1PGDG.rhel7.x86_64.rpm

# For RHEL 8
rpm -ivh --replacepkgs ./third-party/postgresql/libicu-60.3-2.el8_1.x86_64.rpm (only AWS instance)
rpm -ivh --replacepkgs ./third-party/postgresql/postgresql13-libs-13.10-1PGDG.rhel8.x86_64.rpm
rpm -ivh --replacepkgs ./third-party/postgresql/postgresql13-13.10-1PGDG.rhel8.x86_64.rpm
rpm -ivh --replacepkgs ./third-party/postgresql/postgresql13-server-13.10-1PGDG.rhel8.x86_64.rpm

# For RHEL 9
rpm -ivh --replacepkgs ./third-party/postgresql/libicu-67.1-9.el9.x86_64.rpm (only AWS instance)
rpm -ivh --replacepkgs ./third-party/postgresql/postgresql13-libs-13.10-1PGDG.rhel9.x86_64.rpm
rpm -ivh --replacepkgs ./third-party/postgresql/postgresql13-13.10-1PGDG.rhel9.x86_64.rpm
rpm -ivh --replacepkgs ./third-party/postgresql/postgresql13-server-13.10-1PGDG.rhel9.x86_64.rpm
 
chown -R postgres:postgres /var/opt/postgres
 
export PGDATA="/var/opt/postgres/data"
export PGSETUP_INITDB_OPTIONS="-D /var/opt/postgres/data"
 
# For CentOS 7 / RHEL 7,8,9
sed -i "s~^Environment=PGDATA=.*~Environment=PGDATA=/var/opt/postgres/data~" /lib/systemd/system/postgresql-13.service
systemctl daemon-reload
/usr/pgsql-13/bin/postgresql-13-setup initdb
 
Replace "ident" and "peer" with "trust" in postgres hba configuration file, /var/opt/postgres/data/pg_hba.conf. This configuration is for creating the initial users and database, once this is done, change "trust" to md5 in pg_hba.conf and restart PostgreSQL.

  2. Configure PostgreSQL to allow external IP connections.

  3. By default PostgreSQL only allows localhost clients communications. To enable different IPs to communicate with the database you will need to configure the pg_hba.conf file.

    File location according to installation

    • Docker-compose: $JFROG_HOME/xray/var/data/postgres/data

    • Native installations: /var/opt/postgres/data

    To grant all IPs access add the following entry under the IPv4 local connections section.

    host    all             all             0.0.0.0/0               md5

    Add the following entries to /var/opt/postgres/data/postgresql.conf.

    listen_addresses='*'
port=5432

  4. Start PostgreSQL.

    systemctl start postgresql-<version>.service 
or  
service postgresql-<version> start

  5. Setup the database and user.

    ## run the script to seed the tables and schemas needed by Xray
cp -f ./third-party/postgresql/createPostgresUsers.sh /tmp
source /etc/locale.conf
 
cd /tmp && su postgres -c "POSTGRES_PATH=/usr/pgsql-13/bin PGPASSWORD=postgres DB_PASSWORD=password bash /tmp/createPostgresUsers.sh"
Xray PostgreSQL Upgrade

Xray 3.x supports PostgreSQL versions 10, 11, 12, 13, 14, and 15.

To learn about the process and requirements for upgrading PostgreSQL 9.x, see JFrog Xray PostgreSQL Upgrade - PostgreSQL 9.x EoS.

PostgreSQL Performance Improvements

We recommend that you increase the maximum connections setting in the PostgreSQL configuration file.

Open the $JFROG_HOME/xray/var/lib/pgsql/data/postgresql.conf configuration file, and add or edit the max_connections property.

Restart the database to enable this change.

Erlang for Xray
Erlang RPM Installation

Use RPMs with el7 when installing on Centos 7 and RHEL 7, and el8 with CentOS 8 and RHEL8

rpm -ivh --replacepkgs xray/app/third-party/rabbitmq/socat-<version>.rpm
rpm -ivh --replacepkgs xray/app/third-party/rabbitmq/erlang-<version>.rpm
db-util for Xray

db-util allows Xray to interact with the Berkley DB that contains information about RPM-based Docker images. This way, JFrog Xray can index OS packages for these images.

db-util RPM Installation
# This will install db-util if db_dump is not available
hash db_dump 2>/dev/null || rpm -ivh --replacepkgs xray/app/third-party/misc/db4-utils-<version>.rpm