Insight Single Node Manual RPM Installation

JFrog Installation & Setup Documentation

Content Type
Installation & Setup
ft:sourceType
Paligo

The RPM installation bundles Insight and all its dependencies. It is provided as native RPM packages, where Insight and its dependencies must be installed separately. Use this, if you are automating installations.

Before you proceed, see System Requirements for information on supported platforms, supported browsers, and other requirements.

Operating Systems and Platform Support

The following table lists the supported operating systems and the versions.

Product

Debian

RHEL

Ubuntu

Windows Server

Amazon Linux

Insight

10.x, 11.x

8.x, 9.x

20.04, 22.04

Amazon Linux 2023

Operating Systems - End of Support

As part of JFrog commitment to maintain the security and reliability of the JFrog Platform, Artifactory will officially run with Node.js 20.x on all installation types from Artifactory 7.77.3.

Node.js 20.x provided with Linux Archive/Debian/RPM installations (non-containerized distributions) is not supported on the following operating systems.

Hence, these operating systems will no longer supported from Artifactory version 7.77.3.

Supported Platforms

The following table lists the supported platforms.

Product

x86-64

ARM64

Kubernetes

OpenShift

Insight

1.19+

Installation on Kubernetes environments is through Helm Charts. Supported Helm version is Helm 3+.

ARM64 Support

From version 7.41.4, Artifactory supports installation on ARM64 architecture through Helm and Docker installations. You must set up an external database as the Artifactory database since Artifactory does not support the bundled database with the ARM64 installation. Artifactory installation pulls the ARM64 image automatically when you run the Helm or Docker installation on the ARM64 platform.

ARM64 support is also available for Xray, Pipelines (in Helm installation), and Insight. ARM64 support is not available for Distribution.

Database and Third-Party Applications in Insight

A database is required, which is fundamental to management of Insight and is also used to store cluster wide configuration files. Currently PostgreSQL is supported, and any change to configuration requires restarting all Insight nodes for changes to take effect.

Insight supports the following versions of PostgreSQL.

  • 15.x

  • 13.x

  • 12.x

An Elasticsearch database is required to store the data from which Insight generates trends and charts.

Insight supports the following versions of Elasticsearch.

  • 7.17.1 (for Insight 1.11.3 and later)

  • 7.16.2 (for Insight 1.5.0 to 1.10.2)

  • 7.15.1 (for Insight 1.2.3 to 1.30)

  • 7.14.1 (for Insight 1.0.1 to 1.1.3)

Insight Network Ports

Insight uses the 8082 port by default for external communication.

Insight uses the following internal ports by default for communication with JFrog Platform microservices.

  • 8080 for the Insight Server

  • 8082, 8046, 8047, and 8049 for the Router

  • 8087 for Insights

  • 5432 for PostgreSQL (if you use the bundled Postgres database)

  • 9200 for Elasticsearch (if you use the bundled Elasticsearch)

Complete the following steps to install the product.

  1. Download Insight.

  2. Extract the contents of the compressed archive, and go to the extracted folder:

    tar -xvf jfrog-insight-<version>-<compose|rpm|deb>.tar.gz
    cd jfrog-insight-<version>-<compose|rpm|deb>
  3. Install Insight. You must run as a root user.

    rpm -Uvh --replacepkgs ./insight/insight.rpm
  4. Install PostgreSQL and start the PostgreSQL service.

    Already have a PostgreSQL installation?

    Set your PostgreSQL connection details in the Shared Configurations section of the $JFROG_HOME/insight/var/etc/system.yaml file.

    PostgreSQL is required and must be installed before continuing with the next installation steps.

  5. Install Elasticsearch.

    Instructions to install Elasticsearch are available here.

    You can install the package available at jfrog-insight-<version>-rpm/third-party/elasticsearch/elasticsearch-<version>.tar.gzor you can download a compatible version of Elasticsearch from this page.

    When connecting an external instance of Elasticsearch to Insight, add the following flag in the Shared Configurations of $JFROG_HOME/insight/var/etc/system.yaml file.

    shared:  
      elasticsearch:    
       external: true
  6. Install Search Guard.

    The Search Guard package is available at jfrog-insight-<version>-rpm/third-party/elasticsearch/search-guard-<version>.tar.gz. For installation steps, refer to the Search Guard documentation.

    Important

    You must install the Search Guard plugin to ensure secure communication with Elasticsearch.

  7. You must add an admin user to Search Guard to ensure authenticated communication with Elasticsearch. The Search Guard configuration accepts a hashed password.

    Use the following command to generate the hash for the password.

    /etc/elasticsearch/plugins/search-guard-7/tools/hash.sh -p <clear_text_password> 
    #This will output a hashed password (<hash_password>), make a copy of it
  8. Paste the following configuration snippet to add a new (admin) user with the hashed password obtained from the previous step to the end of the /etc/elasticsearch/plugins/search-guard-7/sgconfig/sg_internal_users.yml file.

    <username>:
       hash: "<hashed_password>"
       backend_roles:
         - "admin"
       description: "Insight Elastic admin user"
  9. Enable anonymous access to the _cluster/health endpoint by adding the following snippet to /etc/elasticsearch/plugins/search-guard-7/sgconfig/sg_config.yml file.

    sg_config:
      dynamic:
        http:
          anonymous_auth_enabled: true #set this to true

    This is required to check the health of Elasticsearch cluster.

  10. Map the anonymous user, sg_anonymous , to the backend role, sg_anonymous_backendrole, in the /etc/elasticsearch/plugins/search-guard-7/sgconfig/sg_roles_mapping.yml file.

    sg_anonymous:
      backend_roles:
        - sg_anonymous_backendrole
  11. Add the following snippet to the end of the /etc/elasticsearch/plugins/search-guard-7/sgconfig/sg_roles.yml file.

    sg_anonymous:
      cluster_permissions:
        - cluster:monitor/health
  12. Add the following in the shared section of $JFROG_HOME/insight/var/etc/system.yaml file. Refer to Shared Configurations section.

    shared:
      elasticsearch:
        url: <URL_TO_ELASTICSEARCH_INSTANCE>:<ELASTICSEARCH_PORT>
        username: <USERNAME_SET_IN_SEARCHGUARD>
        password: <CLEAR_TEXT_PASSWORD_FOR_THE_ABOVE_USERNAME>

    If you use Amazon Elasticsearch Service

    Enter the following in the shared section of the YAML file.

    shared:
        elasticsearch:
             url: <URL_TO_ELASTICSEARCH>:<ELASTICSEARCH_PORT>
             external: true
             aes:
                  signed: true
                  serviceName: <AES_SERVICE_NAME>
                  region: <AES_SERVICE_REGION>
                  accessKey: <AWS_ACCESS_KEY>
                  secretKey: <AWS_SECRET_KEY>

    You must log in to the service using your Amazon AWS credentials.

  13. Customize the product configuration.

    1. Set the Artifactory connection details.

    2. Customize the PostgreSQL Database connection details. (optional)

    3. Set any additional configurations (for example: ports, node id) using the Insight system.yaml configuration file.

  14. Start and manage the Insight service.

    systemd OS

    systemctl start|stop insight.service

    systemv OS

    service insight start|stop|status|restart
  15. Access Insight from your browser at: http://<jfrogUrl>/ui/.

    Go to the Dashboard tab in theApplicationmodule in the UI

  16. Check the Insight log.

    tail -f $JFROG_HOME/insight/var/log/console.log

    Configure log rotation of the console log

    The console.log file can grow quickly since all services write to it. For more information, see configure the log rotation.Configuring Log Rotation for Tomcat

After installing and before running Insight, you may set the following configurations.

Where to find the system configurations?

You can configure all your system settings using the system.yaml file located in the $JFROG_HOME/insight/var/etc folder. For more information, see Insight YAML Configuration.

If you don't have a System YAML file in your folder, copy the template available in the folder and name it system.yaml.

Artifactory Connection Details

Insight requires a working Artifactory server and a suitable license. The Insight connection to Artifactory requires 2 parameters:

  • jfrogUrl - URL to the machine where JFrog Artifactory is deployed, or the load balancer pointing to it. We recommend that you use DNS names rather than direct IPs. For example, http://jfrog.acme.com instead of http://10.20.30.40:8082.

    Set it in the Shared Configurations section of the $JFROG_HOME/insight/etc/system.yaml file.

  • join.key - This is the "secret" key required by Artifactory for registering and authenticating the Insight server.

    You can fetch the Artifactory joinKey (join Key) from the JPD UI in the Administration module | User Management | Settings | Join Key.

    Set the join.key used by your Artifactory server in the Shared Configurations section of the $JFROG_HOME/insight/etc/system.yaml file.

Change PostgreSQL Database Credentials

Insight comes bundled with a PostgreSQL Database out-of-the-box, which comes pre-configured with default credentials.

Note

These commands are indicative and assume some familiarity with PostgreSQL. Do not copy and paste them. For docker-compose, you need to ssh into the PostgreSQL container before you run them.

To change the default credentials:

# Access PostgreSQL as the insight user adding the optional -W flag to invoke the password prompt
$ psql -d insight -U insight -W
# Securely change the password for user "mission_control". Enter and then retype the password at the prompt.
\password insight
# Verify the update was successful by logging in with the new credentials
$ psql -d insight -U insight -W
Change Elasticsearch Credentials

Search Guard tool is used to manage authentication. To change password for the default user, Search Guard accepts a hash password to be provided in the configuration.

  1. Obtain the username used to access Elasticsearch from $JFROG_HOME/insight/var/etc/system.yaml available at elasticsearch.username.

  2. Generate the hash password by providing the password(in text format) as input

    $ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/tools/hash.sh -p <password_in_text_format>
  3. Update the configuration of the default user with the output from the previous step.

    vi $ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/sgconfig/sg_internal_users.yml
    #Scroll in the file to find an entry for the username of the default user
    #Update the value for "hash" with the hash content obtained from previous step
    <default_username>:
       hash: <hash_output_from_previous_step>
  4. Run the command to initialise Search Guard.

Add certificates to connect to external Elasticsearch over SSL

To use an external Elasticsearch over an SSL connection, you must copy the certificate files to the trusted folder in the Insight installation ($JFROG_HOME/insight/var/etc/security/keys/trusted) and restart Insight services.

Set your PostgreSQL and Elasticsearch connection details in the Shared Configurations section of the $JFROG_HOME/insight/var/etc/system.yaml file.

Load a custom certificate to Elasticsearch Search Guard

If you prefer to use the custom certificates when Search Guard enabled with tls in Elasticsearch, you can use the search-guard-tlstool to generate Search Guard certificates.

The tool to generate Search Guard certificates is available at $JFROG_HOME/app/third-party/elasticsearch/search-guard-tlstool-1.6.tar.gz. For more information about generating certificates, see Search Guard TLS Tool.

  1. Run the tool to generate the certificates.

    tar -xvf $JFROG_HOME/app/third-party/elasticsearch/search-guard-tlstool-1.6.tar.gz
    cp $JFROG_HOME/app/third-party/elasticsearch/config/tlsconfig.yml $JFROG_HOME/app/third-party/elasticsearch/search-guard-tlstool-1.8/config
    cd $JFROG_HOME/app/third-party/elasticsearch/search-guard-tlstool-1.8/tools
    ./sgtlstool.sh -c ../config/tlsconfig.yml -ca -crt # folder named "out" will be created with all the required certificates,
    cd out
  2. Copy the generated certificates (localhost.key, localhost.pem, root-ca.pem, sgadmin.key, sgadmin.pem ) to the target location based on the installer type.

    Native

    cp localhost.key localhost.pem root-ca.pem sgadmin.key sgadmin.pem  /etc/elasticsearch/certs/
Configure a custom Elasticsearch role

The Search Guard tool is used to manage authentication. By default, an admin user is required to authenticate Elasticsearch. As an alternative to this, you can configure a new user to authenticate Elasticsearch by assigning a custom role with permissions for the application to work.

  1. Add the following snippet to define a new role with custom permissions in $ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/sgconfig/sg_roles.yml.

    <role_name>:
      cluster_permissions:
        - cluster:monitor/health
        - cluster:monitor/main
        - cluster:monitor/state
        - "indices:admin/template/get"
        - "indices:admin/template/delete"
        - "indices:admin/template/put"
        - "indices:admin/aliases"
        - "indices:admin/create"
      index_permissions:
        - index_patterns:
            - "active_*"
          allowed_actions:
            - "indices:monitor/health"
            - "indices:monitor/stats"
            - "indices:monitor/settings/get"
            - "indices:admin/aliases/get"
            - "indices:admin/get"
            - "indices:admin/aliases"
            - "indices:admin/create"
            - "indices:admin/delete"
            - "indices:admin/rollover"
            - SGS_CRUD
  2. Add the following snippet to add a new user in $ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/sgconfig/sg_roles.yml/sg_internal_users.yml.

    <user_name>:
      hash: <Hash_password>
      backend_roles:
        - "<role_name>"   //role_name defined in previous step
      description: "<description>"
  3. Run the following command to generate a hash password.

    $ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/tools/hash.sh -p <clear_text_password>
  4. Add the following snippet to map the new username to the role defined in the previous step in $ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/sgconfig/sg_roles.yml/sg_roles_mapping.yml

    <role_name>:
      users:
        - "<user_name>"
  5. Initialize Search Guard to upload the changes made in the configuration.

  6. Set the new credentials in $JFROG_HOME/insight/etc/system.yaml file:

    shared:
        elasticsearch:
               username: <user_name>
               password: <clear_text_password>
  7. Restart Insight services.

Install PostgreSQL for Insight

You must install PostgreSQL before you proceed with the installation of Insight.

Using Microsoft Azure PostgreSQL

Some managed databases, such as Azure, have a different username for accessing the database than the actual one inside the database. For Azure-managed PostgreSQL, the username will be, for example insight@mycompany and the actualUsername will be insight. For more information, see Insight System YAML.

Passwords for PostgreSQL with special characters

Do not use a password for PostgreSQL that has special characters: Xray may not work if you configure a password that has special characters, such as~ = # @ $ /.

Set up Your PostgreSQL Databases, Users and Schemas

Warning

Database can only be changed for a new installation. Changing the names during an upgrade will result in the loss of existing data.

  1. Log in to the PostgreSQL database as an admin and execute the following commands.

    PostgreSQL Database and User Creation

    CREATE DATABASE insight WITH ENCODING='UTF8' TABLESPACE=pg_default;
    #    Exit from current login
    \q
    #    Login to $DB_NAME database using admin user (by default its postgres)
    psql -U postgres insight
    CREATE USER insight WITH PASSWORD 'password';
    GRANT ALL ON DATABASE insight TO insight;
  2. Configure the system.yaml file with the database configuration details according to the information above. For example.

    shared:
      database:
        type: postgresql
        driver: org.postgresql.Driver
        url: jdbc:postgresql://localhost:5432/insight
        username: insight
        password: password