ARTIFACTORY: Quick Start Guide – RPM

Matthew Wang
2022-01-13 20:39

What will you get?

User-added image

A quick start guide on how to install and configure Artifactory 7 HA for a production ready environment using the RPM/Debian installation. In this guide, we will be walking through setting up a Postgres external database, multiple artifactory nodes, and a NGINX reverse proxy (which can act as a load balancer as well). At minimum, a server per Artifactory node is required. Note that you will need a license per Artifactory node. We also recommend having a type of cloud bucket storage (e.g. S3) ready. A good understanding of linux based operating systems will be required to fully grasp the material throughout this guide.

Before you start

System requirements

Operating system

This installation requires any of the following Linux system versions:

  • Centos 7.x, 8.x 
  • RPM 7.x, 8.x
  • SUSE linux 12 SP 5

Computing environment

Minimum requirement per node: 4-core CPU, 8GB RAM.

We recommend using a machine that is easily scalable in case there is an increase of Artifactory usage.

Load balancer

This example installation uses an NGINX load balancer to control the multiple Artifactory HA nodes.

Ports

The following ports will need to be available internally: 8081, 8082, 8040, 8045, 8048, 9092, 8070, 8086, 8046, 8047, 8049, 8091, 8061, 8062.

Ports 8081 and 8082 will additionally need to be reachable externally between VM/nodes – and your LB/reverse proxy. They do not need to be exposed to the end user.

The other ports just need to be available within the VM and not consumed by another process – some of these ports are configurable via the system.yaml if it cannot be made available.

JFrog license

You will need one Artifactory license for each HA node. 

Install Database

We highly recommend using an external DB with Artifactory – the built in Derby database performance will degrade as you upload more artifacts, and does not support Artifactory HA. We recommend installing your database on a separate server, away from your Artifactory instance, but within minimal latency (same LAN).

Artifactory supports a number of DB types, including mysql, oracle, mariaDB. You may find the links to other supported databases below in the glossary. For more database recommendations, see https://jfrog.com/whitepaper/best-practices-for-managing-your-artifactory-database/ 

In this Quick Start guide, we will cover the Postgres setup.

PostgreSQL Support:

  • 9.5 (EOL)
  • 9.6 (EOL soon)
  • 10.x
  • 11.x
  • 12.x
  • 13.x

External PostgreSQL setup

1. Logged into the DB, run:
CREATE USER artifactory WITH PASSWORD 'password';
CREATE DATABASE artifactory WITH OWNER=artifactory ENCODING='UTF8';
GRANT ALL PRIVILEGES ON DATABASE artifactory TO artifactory;
2. Download the JDBC driver corresponding to your PostgreSQL version from the  PostgreSQL JDBC Driver Download site and copy the downloaded jar file into $JFROG_HOME/artifactory/var/bootstrap/artifactory/tomcat/lib directory. Make sure your driver has read permissions for all users. Note that this directory will get created on installation of the RPM/Debin, or you may pre-create it.
3. Add the database connection details in the system.yaml configuration file, under $JFROG_HOME/artifactory/var/etc/system.yaml 
 shared:
database:
type: postgresql
driver: org.postgresql.Driver
url: jdbc:postgresql://<your db url, for example: localhost:5432>/artifactory
username: artifactory
password: password

Make sure to keep your YAML spacing consistent! We recommend either 2 or 4 spaces indentation, but ensure that it is consistent throughout.

4. Enabling PostgreSQL connectivity from the Artifactory servers:

a. Add the following line to  <postgres_mount>/data/pg_hba.conf.host artifactory artifactory <artifactory_ip> md5

b. Add the following line to (if it already exists, update it) <postgres_mount>/data/postgresql.conf 

listen_addresses='*'

We recommend having sizable storage space for the database, even if it is only storing metadata. It should be at least 2/10 of your expected file store size.

Configure filestore

The binary storage is configurable in the `$JFROG_HOME/var/etc/artifactory/binarystore.xml`. Below is a sample configuration using an Amazon S3 bucket. Note that this file/directory will get created on installation of the RPM/Debin, or you may pre-create it.

Below is a sample S3v3 HA configuration

<config version="2">
<chain template="cluster-s3-storage-v3"/>
<provider id="s3-storage-v3" type="s3-storage-v3">
<endpoint>s3.amazonaws.com</endpoint>
<bucketName>bucketName</bucketName>
<path>pathPrefix</path>
<region>s3Region</region>
<identity>yourIdentity</identity>
<credential>yourCredentials</credential>
<usePresigning>true</usePresigning>
<signatureExpirySeconds>600</signatureExpirySeconds>
</provider>
</config>

Steps to Install

1. Install Artifactory as a service on Red Hat compatible Linux distributions, as a root user.yum install -y jfrog-artifactory-<pro|oss|cpp-ce>-<version>.rpm

2. Enable HA in system.yaml (and add the DB details if not done already):shared:
extraJavaOpts: "-Xms512m -Xmx4g"
node:
haEnabled: true
taskAffinity: any
On additional nodes, copy over $JFROG_HOME/var/etc/security/master.key from the first node to the same location. The first node will automatically generate this file on first start up. Again, make sure to be consistent with the spacing in system.yaml.  If you would like to generate your own key ahead of time, you can follow the guide here to do so.

3. Configure file store in $JFROG_HOME/var/etc/artifactory/binarystore.xml. See https://www.jfrog.com/confluence/display/JFROG/Configuring+the+Filestore#ConfiguringtheFilestore-AmazonS3SDKClusterBinaryProvider. We will use the Amazon SDK provider in this guide.<config version="2">
<chain template="cluster-s3-storage-v3"/>
<provider id="s3-storage-v3" type="s3-storage-v3">
<endpoint>s3.amazonaws.com</endpoint>
<bucketName>bucketName</bucketName>
<path>pathPrefix</path>
<region>s3Region</region>
<identity>yourIdentity</identity>
<credential>yourCredentials</credential>
<usePresigning>true</usePresigning>
<signatureExpirySeconds>600</signatureExpirySeconds>
</provider>
</config>

4. Manage Artifactory using the following commands:service artifactory start|stop5. Check the console.log for the following printout of start-up successtail -f $JFROG_HOME/artifactory/var/log/console.log
2021-09-20T18:25:44.992Z [jfrou] [INFO ] [470978b404ac5eac] [local_topology.go:270 ] [main ] -
###############################################################
### All services started successfully in 52.558 seconds ###
###############################################################

6. Once Artifactory comes up, the UI should be accessible at port 8082. Check that Artifactory is in HA mode by running the following REST API and look in the addon array for “ha”:

$ curl localhost:8082/artifactory/api/system/version -u admin:password
{
"version" : "7.25.7",
"revision" : "72507900",
"addons" : [ "ha",...
If it is missing, it means Artifactory did not start in HA mode. Alternatively, the Artifactory-service.log will also print out an ASCII art of “Artifactory HA” during start-up. It will print “Artifactory Pro” if it isn’t in HA mode. 

7. For a new node to join a cluster, the nodes must connect to the same database and have the same Master Key. For additional node(s), repeat steps  1 – 6. Note that you will need a license per Artifactory node.

Post-Install Steps

SSL

Enabling SSL can be done using a reverse proxy or load balancer such as Nginx, Apache, HAProxy, F5, etc… Artifactory has a built-in configuration generator for Nginx and Apache in the UI under Admin -> Http Settings. You can place the configuration in the conf.d folder under /etc/nginx or /etc/httpd.

###########################################################
## this configuration was generated by JFrog Artifactory ##
###########################################################

## add ssl entries when https has been set in config
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_certificate /etc/ssl/private/server.key;
ssl_certificate_key /etc/ssl/private/server.crt;
ssl_session_cache shared:SSL:1m;
ssl_prefer_server_ciphers on;
## server configuration
server {
listen 443 ssl;
listen 80 ;
server_name ~(?<repo>.+)\.artifactory_host artifactory_host;

if ($http_x_forwarded_proto = '') {
set $http_x_forwarded_proto $scheme;
}
## Application specific logs
## access_log /var/log/nginx/artifactory_host-access.log timing;
## error_log /var/log/nginx/artifactory_host-error.log;
rewrite ^/$ /ui/ redirect;
rewrite ^/ui$ /ui/ redirect;
rewrite ^/(v1|v2)/(.*) /artifactory/api/docker/$repo/$1/$2;
chunked_transfer_encoding on;
client_max_body_size 0;
location / {
proxy_read_timeout 2400s;
proxy_pass_header Server;
proxy_cookie_path ~*^/.* /;
proxy_buffer_size 128k;
proxy_buffers 40 128k;
proxy_busy_buffers_size 128k;
proxy_pass http://localhost:8082;
proxy_set_header X-JFrog-Override-Base-Url $http_x_forwarded_proto://$host:$server_port;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

location ~ ^/artifactory/ {
proxy_pass http://localhost:8081;
}
}
}
Reverse proxies can also be configured to handle load balancing traffic between nodes. In NGINX’s case, you can add this snippet at the top: upstream artifactory {
server 10.150.0.222:8082;
server <additional node IP>:8082;
}

upstream artifactory-direct {
server 10.150.0.222:8081;
server <additional node IP>:8081;
}
Then, modify the proxy_pass lines to be:proxy_pass http://localhost:8082; → proxy_pass http://artifactory;
proxy_pass http://localhost:8081; → proxy_pass http://artifactory-direct;

Otherwise, you may use a dedicated load balancer to handle traffic balancing. 

Note that to support docker requests, you’ll need a reverse proxy or load balancer to handle request rewrites. Also, if you are planning on having a load balancer terminating SSL, and a reverse proxy, you’ll need the below headers to be hard coded to the details of your load balancer.

For NGINX:proxy_set_header X-JFrog-Override-Base-Url https://<LBHOST>:<LBPORT>;
proxy_set_header X-Forwarded-Port <LBPORT>
proxy_set_header X-Forwarded-Proto https

Tuning Artifactory (Optional)

We have the following optional tuning section to optimize Artifactory for heavier loads – it is a good idea to keep these parameters in mind as your Artifactory instance takes on more load. 

Heap size

javaOpts (heap size) in system.yaml – we recommend at least setting this one:shared:
extraJavaOpts: "-Xms512m -Xmx4g"

Database connections

To customize database connections in system.yaml:artifactory:
database:
maxOpenConnections: 200

access:
database:
maxOpenConnections: 200

metadata:
database:
maxOpenConnections: 200
As a rule of thumb, we require (upto) a number of DB connections based on the following formula:
Total number of connections = (number of nodes) * ((artifactory.database.maxOpenConnections * 2) + access.database.maxOpenConnections + metadata.database.maxOpenConnections) + 50

Tomcat threads

Tune Tomcat threads in system.yaml:artifactory:
tomcat:
connector:
maxThreads: 400

access:
tomcat:
connector:
maxThreads: 100
When modifying the Access maxThreads, it is required to update the $JFROG_HOME/artifactory/var/etc/artifactory/artifactory.system.properties file with:

artifactory.access.client.max.connections = <VALUE>

Async thread pool

Tune Async thread pool in the same file. Note the corePoolSize should not be more than 8x the number of CPU cores:artifactory.async.corePoolSize = 32
artifactory.async.poolMaxQueueSize = 100000

Upgrade Steps

RPM Upgrade:

1. Remove the first node from the load balancer. All requests will be directed to the additional nodes. Check the $HOME/logs/request.log and ARTIFACTORY_URL/api/tasks (search for "running") to ensure that Artifactory is completely inactive.

2. In the target cluster, keep only one node running, and perform a graceful shutdown to the rest of the nodes.
service artifactory stop

3. Install Artifactory as a service on Red Hat compatible Linux distributions on stopped nodes, as a root user.
yum -y install jfrog-artifactory-<pro|oss|cpp-ce>-<version>

4. Check that the migration has completed successfully, by reviewing the following files:
migration log: $JFROG_HOME/artifactory/var/log/migration.log
system.yaml configuration: $JFROG_HOME/artifactory/var/etc/system.yaml

This newly created file will contain your current custom configurations in the new format.

5. Manage Artifactory.
service artifactory start|stop

6. Check the console.log for the following printout of start-up successtail -f $JFROG_HOME/artifactory/var/log/console.log
2021-09-20T18:25:44.992Z [jfrou] [INFO ] [470978b404ac5eac] [local_topology.go:270 ] [main ] -
###############################################################
### All services started successfully in 52.558 seconds ###
###############################################################
7. Once all nodes except running node are upgraded, perform steps 2 – 6 on the last node

Glossary

Download link: https://jfrog.com/download-jfrog-platform/
How to Manage the DB: https://jfrog.com/whitepaper/best-practices-for-managing-your-artifactory-database/
System Requirements: https://www.jfrog.com/confluence/display/JFROG/System+Requirements
Using External Databases:
https://www.jfrog.com/confluence/display/JFROG/Oracle 
https://www.jfrog.com/confluence/display/JFROG/MySQL 
https://www.jfrog.com/confluence/display/JFROG/Microsoft+SQL+Server 
https://www.jfrog.com/confluence/display/JFROG/PostgreSQL 
https://www.jfrog.com/confluence/display/JFROG/MariaDB 
Steps to Install Artifactory:
https://www.jfrog.com/confluence/display/JFROG/Installing+Artifactory#InstallingArtifactory-RPMInstallation
Tune Artifactory: https://jfrog.com/knowledge-base/how-do-i-tune-artifactory-for-heavy-loads/
Filestore setup with s3: https://www.jfrog.com/confluence/display/JFROG/Configuring+the+Filestore#ConfiguringtheFilestore-AmazonS3SDKClusterBinaryProvider
RPM Upgrade: https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory#UpgradingArtifactory-RPMUpgrade.1 
Debian Upgrade: 
https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory#UpgradingArtifactory-DebianUpgrade.1 
Reverse proxy and Docker rewrites:
https://www.jfrog.com/confluence/display/JFROG/Getting+Started+with+Artifactory+as+a+Docker+Registry 
https://www.jfrog.com/confluence/display/JFROG/HTTP+Settings