Insight Single Node Manual Debian Installation

The Debian installation bundles Insight and all its dependencies. They are provided as native Debian packages, where you must separately install Insight and its dependencies. Use this if you are automating installations.

Before you proceed, see System Requirements for information on supported platforms, supported browsers, and other requirements.

Insight System Requirements

Operating Systems and Platform Support

The following table lists the supported operating systems and the versions.

Product	Debian	Centos*	RHEL	Ubuntu	Windows Server	SLES	Amazon Linux
Insight	10.x, 11.x	7.x	7.x, 8.x	18.04, 20.04, 22.04		12 SP5	Amazon Linux 2023, Amazon Linux 2

CentOS 8.x Support

CentOS 8.x reached its end-of-life in December 2021. CentOS 8.x support for JFrog products has been deprecated by the end of June 2022.

Supported Platforms

The following table lists the supported platforms.

Product	x86-64	ARM64	Kubernetes	OpenShift
Insight			1.19+

Installation on Kubernetes environments is through Helm Charts. Supported Helm version is Helm 3+.

ARM64 Support

From version 7.41.4, Artifactory supports installation on ARM64 architecture through Helm and Docker installations. You must set up an external database as the Artifactory database since Artifactory does not support the bundled database with the ARM64 installation. Artifactory installation pulls the ARM64 image automatically when you run the Helm or Docker installation on the ARM64 platform.

ARM64 support is also available for Xray, Distribution, and Insight. ARM64 support is not available for Pipelines.

Database and Third-Party Applications in Insight

A database is required, which is fundamental to management of Insight and is also used to store cluster wide configuration files. Currently PostgreSQL is supported, and any change to configuration requires restarting all Insight nodes for changes to take effect.

Insight supports the following versions of PostgreSQL.

13.x
12.x
11.x
10.x

An Elasticsearch database is required to store the data from which Insight generates trends and charts.

Insight supports the following versions of Elasticsearch.

7.17.1 (for Insight 1.11.3 and later)
7.16.2 (for Insight 1.5.0 to 1.10.2)
7.15.1 (for Insight 1.2.3 to 1.30)
7.14.1 (for Insight 1.0.1 to 1.1.3)

Insight Network Ports

Insight uses the 8082 port by default for external communication.

Insight uses the following internal ports by default for communication with JFrog Platform microservices.

8080 for the Insight Server
8082, 8046, 8047, and 8049 for the Router
8087 for Insights
5432 for PostgreSQL (if you use the bundled Postgres database)
9200 for Elasticsearch (if you use the bundled Elasticsearch)

Complete the following steps to install the product.

Download Insight.
Extract the contents of the compressed archive, and go to the extracted folder.
```
tar -xvf jfrog-insight-<version>-deb.tar.gz
cd jfrog-insight-<version>-deb
```
Install Insight. You must run as a root user.
```
dpkg -i ./insight/insight.deb
```
Install PostgreSQL for InsightPosgreSQL and start the PostgreSQL service.
Already have a PostgreSQL installation?
Set your PostgreSQL connection details in the Shared Configurations section of the $JFROG_HOME/insight/var/etc/system.yaml file.
PostgreSQL is required and must be installed before continuing with the next installation steps.
Install Elasticsearch.
Instructions to install Elasticsearch are available here.
You can install the package available at jfrog-insight-<version>-deb/third-party/elasticsearch/elasticsearch-<version>.tar.gzor you can download a compatible version of Elasticsearch from this page.
When connecting an external instance of Elasticsearch to Insight, add the following flag in the Shared Configurations of $JFROG_HOME/insight/var/etc/system.yaml file.
```
shared:  
  elasticsearch:    
   external: true
```
Install Search Guard.
The Search Guard package is available at jfrog-insight-<version>-deb/third-party/elasticsearch/search-guard-<version>.tar.gz. For installation steps, refer to the Search Guard documentation.
Important
You must install the Search Guard plugin to ensure secure communication with Elasticsearch.
You must add an admin user to Search Guard to ensure authenticated communication with Elasticsearch. The Search Guard configuration accepts a hashed password.
Use the following command to generate the hash for the password.
```
/etc/elasticsearch/plugins/search-guard-7/tools/hash.sh -p <clear_text_password> 
#This will output a hashed password (<hash_password>), make a copy of it
```
Paste the following configuration snippet to add a new (admin) user with the hashed password obtained from the previous step to the end of the /usr/share/elasticsearch/plugins/search-guard-7/sgconfig/sg_internal_users.yml file.
```
<username>:
   hash: "<hashed_password>"
   backend_roles:
     - "admin"
   description: "Insight Elastic admin user"
```
Enable anonymous access to the _cluster/health endpoint by adding the following snippet to /usr/share/elasticsearch/plugins/search-guard-7/sgconfig/sg_config.yml file.
```
sg_config:
  dynamic:
    http:
      anonymous_auth_enabled: true #set this to true
```
This is required to check the health of Elasticsearch cluster.
Map the anonymous user, sg_anonymous , to the backend role, sg_anonymous_backendrole, in the /usr/share/elasticsearch/plugins/search-guard-7/sgconfig/sg_roles_mapping.yml file.
```
sg_anonymous:
  backend_roles:
    - sg_anonymous_backendrole
```
Add the following snippet to the end of the /usr/share/elasticsearch/plugins/search-guard-7/sgconfig/sg_roles.yml file.
```
sg_anonymous:
  cluster_permissions:
    - cluster:monitor/health
```

Add the following in the shared section of$JFROG_HOME/insight/var/etc/system. yaml file. Refer to Shared Configurations section.

shared:
  elasticsearch:
    url: <URL_TO_ELASTICSEARCH_INSTANCE>:<ELASTICSEARCH_PORT>
    username: <USERNAME_SET_IN_SEARCHGUARD>
    password: <CLEAR_TEXT_PASSWORD_FOR_THE_ABOVE_USERNAME>

If you use Amazon Elasticsearch Service

Enter the following in the shared section of the YAML file.

shared:
    elasticsearch:
         url: <URL_TO_ELASTICSEARCH>:<ELASTICSEARCH_PORT>
         external: true
         aes:
              signed: true
              serviceName: <AES_SERVICE_NAME>
              region: <AES_SERVICE_REGION>
              accessKey: <AWS_ACCESS_KEY>
              secretKey: <AWS_SECRET_KEY>

You must log in to the service using your Amazon AWS credentials.

Customize the product configuration.
1. Set the Artifactory connection details.
2. Customize the PostgreSQL Database connection details. (optional)
3. Set any additional configurations (for example: ports, node id) using the Insight system.yaml configuration file.

Start and manage the Insight service.

systemd OS

systemctl start|stop insight.service

systemv OS

service insight start|stop|status|restart

Access Insight from your browser at:http://<jfrogUrl>/ui/.
Go to theDashboard tab in theApplicationmodule in the UI.
Check the Insight log.
```
tail -f $JFROG_HOME/insight/var/log/console.log
```
Configure log rotation of the console log
The console.log file can grow quickly since all services write to it. For more information, see configure the log rotation.Logging

Insight Product Configuration

After installing and before running Insight, you may set the following configurations.

Where to find the system configurations?

You can configure all your system settings using the system.yaml file located in the $JFROG_HOME/insight/var/etc folder. For more information, see Insight YAML Configuration.

If you don't have a System YAML file in your folder, copy the template available in the folder and name it system.yaml.

Artifactory Connection Details

Insight requires a working Artifactory server and a suitable license. The Insight connection to Artifactory requires 2 parameters:

jfrogUrl - URL to the machine where JFrog Artifactory is deployed, or the load balancer pointing to it. We recommend that you use DNS names rather than direct IPs. For example, http://jfrog.acme.com instead of http://10.20.30.40:8082.
Set it in the Shared Configurations section of the $JFROG_HOME/insight/etc/system.yaml file.
join.key - This is the "secret" key required by Artifactory for registering and authenticating the Insight server.
You can fetch the Artifactory joinKey (join Key) from the JPD UI in the Administration module | User Management | Settings | Join Key.
Set the join.key used by your Artifactory server in the Shared Configurations section of the $JFROG_HOME/insight/etc/system.yaml file.

Change PostgreSQL Database Credentials

Insight comes bundled with a PostgreSQL Database out-of-the-box, which comes pre-configured with default credentials.

Note

These commands are indicative and assume some familiarity with PostgreSQL. Do not copy and paste them. For docker-compose, you need to ssh into the PostgreSQL container before you run them.

To change the default credentials:

# Access PostgreSQL as the insight user adding the optional -W flag to invoke the password prompt
$ psql -d insight -U insight -W
# Securely change the password for user "mission_control". Enter and then retype the password at the prompt.
\password insight
# Verify the update was successful by logging in with the new credentials
$ psql -d insight -U insight -W

Change Elasticsearch Credentials

Search Guard tool is used to manage authentication. To change password for the default user, Search Guard accepts a hash password to be provided in the configuration.

Obtain the username used to access Elasticsearch from $JFROG_HOME/insight/var/etc/system.yaml available at elasticsearch.username.

Generate the hash password by providing the password(in text format) as input

$ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/tools/hash.sh -p <password_in_text_format>

Update the configuration of the default user with the output from the previous step.

vi $ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/sgconfig/sg_internal_users.yml
#Scroll in the file to find an entry for the username of the default user
#Update the value for "hash" with the hash content obtained from previous step
<default_username>:
   hash: <hash_output_from_previous_step>

Run the command to initialise Search Guard.

Add certificates to connect to external Elasticsearch over SSL

To use an external Elasticsearch over an SSL connection, you must copy the certificate files to the trusted folder in the Insight installation ($JFROG_HOME/insight/var/etc/security/keys/trusted) and restart Insight services.

Set your PostgreSQL and Elasticsearch connection details in the Shared Configurations section of the $JFROG_HOME/insight/var/etc/system.yaml file.

Load a custom certificate to Elasticsearch Search Guard

If you prefer to use the custom certificates when Search Guard enabled with tls in Elasticsearch, you can use the search-guard-tlstool to generate Search Guard certificates.

The tool to generate Search Guard certificates is available at $JFROG_HOME/app/third-party/elasticsearch/search-guard-tlstool-1.6.tar.gz. For more information about generating certificates, see Search Guard TLS Tool.

Run the tool to generate the certificates.

tar -xvf $JFROG_HOME/app/third-party/elasticsearch/search-guard-tlstool-1.6.tar.gz
cp $JFROG_HOME/app/third-party/elasticsearch/config/tlsconfig.yml $JFROG_HOME/app/third-party/elasticsearch/search-guard-tlstool-1.8/config
cd $JFROG_HOME/app/third-party/elasticsearch/search-guard-tlstool-1.8/tools
./sgtlstool.sh -c ../config/tlsconfig.yml -ca -crt # folder named "out" will be created with all the required certificates,
cd out

Copy the generated certificates (localhost.key, localhost.pem, root-ca.pem, sgadmin.key, sgadmin.pem ) to the target location based on the installer type.
Native
```
cp localhost.key localhost.pem root-ca.pem sgadmin.key sgadmin.pem  /etc/elasticsearch/certs/
```

Configure a custom Elasticsearch role

The Search Guard tool is used to manage authentication. By default, an admin user is required to authenticate Elasticsearch. As an alternative to this, you can configure a new user to authenticate Elasticsearch by assigning a custom role with permissions for the application to work.

Add the following snippet to define a new role with custom permissions in $ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/sgconfig/sg_roles.yml.

<role_name>:
  cluster_permissions:
    - cluster:monitor/health
    - cluster:monitor/main
    - cluster:monitor/state
    - "indices:admin/template/get"
    - "indices:admin/template/delete"
    - "indices:admin/template/put"
    - "indices:admin/aliases"
    - "indices:admin/create"
  index_permissions:
    - index_patterns:
        - "active_*"
      allowed_actions:
        - "indices:monitor/health"
        - "indices:monitor/stats"
        - "indices:monitor/settings/get"
        - "indices:admin/aliases/get"
        - "indices:admin/get"
        - "indices:admin/aliases"
        - "indices:admin/create"
        - "indices:admin/delete"
        - "indices:admin/rollover"
        - SGS_CRUD

Add the following snippet to add a new user in $ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/sgconfig/sg_roles.yml/sg_internal_users.yml.
```
<user_name>:
  hash: <Hash_password>
  backend_roles:
    - "<role_name>"   //role_name defined in previous step
  description: "<description>"
```

Run the following command to generate a hash password.

$ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/tools/hash.sh -p <clear_text_password>

Add the following snippet to map the new username to the role defined in the previous step in $ELASTICSEARCH_HOME/plugins/search-guard-<major_version_number>/sgconfig/sg_roles.yml/sg_roles_mapping.yml
```
<role_name>:
  users:
    - "<user_name>"
```
Initialize Search Guard to upload the changes made in the configuration.

Set the new credentials in $JFROG_HOME/insight/etc/system.yaml file:

shared:
    elasticsearch:
           username: <user_name>
           password: <clear_text_password>

Restart Insight services.

Install PostgreSQL for Insight

You must install PostgreSQL before you proceed with the installation of Insight.

Using Microsoft Azure PostgreSQL

Some managed databases, such as Azure, have a different username for accessing the database than the actual one inside the database. For Azure-managed PostgreSQL, the username will be, for example insight@mycompany and the actualUsername will be insight. For more information, see Insight System YAML.

Passwords for PostgreSQL with special characters

Do not use a password for PostgreSQL that has special characters: Xray may not work if you configure a password that has special characters, such as~ = # @ $ /.

We recommend that you to ensure your apt-get libraries are up-to-date, using the following commands.

apt-get update
apt-get install -f -y
apt-get update

Create the file repository configuration to pull PostgreSQL dependencies..

cp -f /etc/apt/sources.list /etc/apt/sources.list.origfile
sh -c 'echo "deb http://ftp.de.debian.org/debian/ $(lsb_release -cs) main non-free contrib" >> /etc/apt/sources.list'
sh -c 'echo "deb-src http://ftp.de.debian.org/debian/ $(lsb_release -cs) main non-free contrib" >> /etc/apt/sources.list' 
    
cp -f /etc/apt/sources.list.d/pgdg.list /etc/apt/sources.list.d/pgdg.list.origfile
sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" >> /etc/apt/sources.list.d/pgdg.list'
sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" >> /etc/apt/sources.list.d/pgdg.list'
  
wget --no-check-certificate --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

Install PostgreSQL.

Run the following commands from the extracted jfrog-insight-<version>-deb directory.

mkdir -p /var/opt/postgres/data

Ubuntu 16.04 (xenial)

dpkg -i ./third-party/postgresql/postgresql-13_13.2-1.pgdg16.04+1_amd64.deb

Ubuntu 18.04 (bionic)

dpkg -i ./third-party/postgresql/postgresql-13_13.2-1.pgdg18.04+1_amd64.deb

Ubuntu 20.04 (focal)

dpkg -i ./third-party/postgresql/postgresql-13_13.2-1.pgdg20.04+1_amd64.deb

Debian 8 (jessie)

## Before installing Postgres dependencies
mv /etc/apt/sources.list.d/backports.list /etc/apt >/dev/null
apt-get update
dpkg -i ./third-party/postgresql/postgresql-13_13.2-1.pgdg80+1_amd64.deb
 
# After installing Postgres dependencies
mv /etc/apt/backports.list /etc/apt/sources.list.d/backports.list >/dev/null
apt-get update

Debian 9 (stretch)

dpkg -i ./third-party/postgresql/postgresql-13_13.2-1.pgdg90+1_amd64.deb

Debian 10 (buster)

apt update -y
apt-get install wget sudo -y
apt-get install -y gnupg gnupg1 gnupg2
dpkg -i ./third-party/postgresql/postgresql-13_13.2-1.pgdg100+1_amd64.deb

Stop the PostgreSQL service.
```
systemctl stop postgresql.service
```

Change permissions for the PostgreSQL folder.

chown -R postgres:postgres /var/opt/postgres
 
sed -i "s~^data_directory =.*~data_directory = '/var/opt/postgres/data'~" "/etc/postgresql/13/main/postgresql.conf"
sed -i "s~^hba_file =.*~hba_file = '/var/opt/postgres/data/pg_hba.conf'~" "/etc/postgresql/13/main/postgresql.conf"
sed -i "s~^ident_file =.*~ident_file = '/var/opt/postgres/data/pg_ident.conf'~" "/etc/postgresql/13/main/postgresql.conf"
 
su postgres -c "/usr/lib/postgresql/13/bin/initdb --pgdata=/var/opt/postgres/data"

Configure PostgreSQL to allow external IP connections.
By default, PostgreSQL only allows localhost clients communications. To enable different IPs to communicate with the database you need to configure the pg_hba.conf file.
The file is available at /var/opt/postgres/data.
To grant all IPs access, add the following entry under the IPv4 local connections section.
```
host    all             all             0.0.0.0/0               trust
```
Add the following line to /var/opt/postgres/data/postgresql.conf.
```
listen_addresses='*'
port=5432
```

Start PostgreSQL.

systemctl start postgresql-13.service 
 
or 
 
service postgresql-13 start

Setup the database and user.

Run the following commands to seed the tables and schema.

sudo -u postgres psql -c "CREATE USER insight WITH PASSWORD 'password';"
sudo -u postgres psql -c "CREATE DATABASE insight WITH OWNER=insight ENCODING='UTF8';"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE insight TO insight;"

cp -f ./third-party/postgresql/createPostgresUsers.sh /tmp
source /etc/locale.conf
cd /tmp && su postgres -c "POSTGRES_PATH=/usr/pgsql-13/bin PGPASSWORD=postgres DB_PASSWORD=password bash /tmp/createPostgresUsers.sh"

Put back the original pgdg.list.

mv /etc/apt/sources.list.d/pgdg.list /etc/apt/sources.list.d/pgdg.list.tmp &&
cp -f /etc/apt/sources.list.d/pgdg.list.origfile /etc/apt/sources.list.d/pgdg.list

Remove backup files.

rm -f /etc/apt/sources.list.d/pgdg.list.tmp
rm -f /etc/apt/sources.list.d/pgdg.list.origfile

Put back the originalsources.list .

mv /etc/apt/sources.list /etc/apt/sources.list.tmp &&
cp -f /etc/apt/sources.list.origfile /etc/apt/sources.list

Remove the backup files.

rm -f /etc/apt/sources.list.tmp &&
rm -f /etc/apt/sources.list.origfile

Set up Your PostgreSQL Databases, Users and Schemas

Warning

Database can only be changed for a new installation. Changing the names during an upgrade will result in the loss of existing data.

PostgreSQL Database and User Creation

CREATE DATABASE insight WITH ENCODING='UTF8' TABLESPACE=pg_default;
#    Exit from current login
\q
#    Login to $DB_NAME database using admin user (by default its postgres)
psql -U postgres insight
CREATE USER insight WITH PASSWORD 'password';
GRANT ALL ON DATABASE insight TO insight;

Configure the system.yaml file with the database configuration details according to the information above. For example.

shared:
  database:
    type: postgresql
    driver: org.postgresql.Driver
    url: jdbc:postgresql://localhost:5432/insight
    username: insight
    password: password