ARTIFACTORY: Docker Quick Start Guide

Loren Yeung
2022-01-13 20:36

What will you get?

This guide will outline how to install Artifactory 7 HA for a production ready environment via the docker installation. In this guide, we will be walking through setting up a Postgres external database, multiple artifactory nodes, and a NGINX reverse proxy (which can act as a load balancer as well). At minimum, a server per Artifactory node is required. Note that you will need a license per Artifactory node. We also recommend having a type of cloud bucket storage (e.g. S3) ready. A good understanding of linux based operating systems and docker will be required to fully grasp the material throughout this guide.

User-added image

Pre-requisites

Minimum Docker engine v18.

For the docker installation, setting up a mount point is important. This will cover the Artifactory home and allow configurations to persist. 

OS that supports docker engine v18
https://docs.docker.com/engine/install/

The following ports will need to be available internally (within the container): 8081, 8082, 8040, 8045, 8048, 9092, 8070, 8086, 8046, 8047, 8049, 8091, 8061, 8062. 8081 and 8082 will additionally need to be reachable externally between nodes – and your LB/reverse proxy. These two ports will be exposed below during container start up. They do not need to be exposed to the end user. The other ports just need to be available within the container and not consumed by another process – some of these ports are configurable via the system.yaml if it cannot be made available.

Environment details

Before installing Artifactory, refer to System Requirements for information on supported platforms, supported browsers and other requirements. We recommend using a machine that is easily scalable in case there is an increase of Artifactory usage. Generally a machine with 4-core CPU, 8GB RAM per node is enough starting out.
See for more node hardware details https://www.jfrog.com/confluence/display/JFROG/System+Requirements 

Pre-setup

1. Create a directory for your Artifactory home. For my example, I chose to create a ‘jfrog’ directory:

/home/loreny/jfrog

2. Within the jfrog directory, create a directory tree:

$ mkdir -p artifactory/var/etc/
$ mkdir -p artifactory/var/bootstrap/artifactory/tomcat/lib

3. Create a system.yaml

$ touch jfrog/artifactory/var/etc/system.yaml

4. This file covers most configurations that you’ll be tweaking for Artifactory, and our other products. We’ll be editing it continually below. Retrieve the host machine’s IP, and give it a unique ID. If you do not set these, other nodes cannot reach Artifactory. This will go into the system.yaml like so:

shared:
extraJavaOpts: "-Xms512m -Xmx4g"
node:
ip: 10.150.0.222
id: nodeone
haEnabled: true
taskAffinity: any

Make sure to keep your YAML spacing consistent! We recommend either 2 or 4 spaces indentation, but ensure that it is consistent throughout.
5. Provide the whole directory with proper permissions; by default this just means ownership by the 1030 user (default artifactory id). You do not need to explicitly create a user with the id 1030 to run the chown. You will need to have sudo level permissions to run the chown successfully.
 

$ chown -R 1030:1030 jfrog/

Database

We highly recommend using an external DB with Artifactory – the built in derby database performance will degrade as you upload more artifacts, and does not support Artifactory HA. We recommend installing your database on a separate server, away from your Artifactory instance, but within minimal latency (same LAN). In this quickstart guide, we will cover the Postgres setup.

PostgreSQL support:

  • 9.5 (EOL)
  • 9.6 (EOL soon)
  • 10.x
  • 11.x
  • 12.x
  • 13.x

External Postgres setup:

1. Log on to the DB, run the following SQL statements:CREATE USER artuser WITH PASSWORD 'PassJFrog!453';
CREATE DATABASE artifactory WITH OWNER=artuser ENCODING='UTF8';
GRANT ALL PRIVILEGES ON DATABASE artifactory TO artuser;
2. Download the JDBC driver corresponding to your PostgreSQL version from the PostgreSQL JDBC Driver Download site and copy the downloaded jar file into artifactory/var/bootstrap/artifactory/tomcat/lib directory. Make sure your driver has read permissions for all users. 
3. Adjust the database connection details in the system.yaml configuration file. For purposes of demonstration, my DB will be accessible at 10.150.0.223 on port 5432.
shared:type: postgresql
driver: org.postgresql.Driver
url: jdbc:postgresql://10.150.0.223:5432/artifactory
username: artuser
password: PassJFrog!453
4. Enabling PostgreSQL connectivity from the Artifactory servers:
Add the following line to  <postgres_mount>/data/pg_hba.conf.host artifactory artifactory <artifactory_ip> md5Add the following line to (if it already exists, update it) <postgres_mount>/data/postgresql.conf
listen_addresses='*'
Artifactory supports a number of DB types, including mysql, oracle, mariaDB. You may find the links to other supported databases below in the glossary.

We recommend having sizable storage space for the database, even if it is only storing metadata. It should be at least 2/10 of your expected file store size.

For more database recommendations, see Best Practices for Managing Your Artifactory Database  

File store (S3/Local):

Artifactory stores binaries separately from the database. By default, this is done on the same filesystem as Artifactory’s installation. For ease of scaling, we recommend setting up a s3 bucket and utilizing that for unlimited scaling. The file store is configured via the file binarystore.xml, which lives under:

artifactory/var/etc/artifactory/binarystore.xml.

To configure it to use S3, you’ll want to use the following template and fill out your details :

<config version="2">
<chain template="cluster-s3-storage-v3"/>
<provider id="s3-storage-v3" type="s3-storage-v3">
<endpoint>s3.amazonaws.com</endpoint>
<bucketName>bucketName</bucketName>
<path>pathPrefix</path>
<region>s3Region</region>
<identity>yourIdentity</identity>
<credential>yourCredentials</credential>
<usePresigning>true</usePresigning>
<signatureExpirySeconds>600</signatureExpirySeconds>
</provider>
</config>

Configuring the Filestore – JFrog
We have other configuration templates for other cloud providers as well:
Configuring the Filestore – JFrog

Local

If you want to use local disk storage instead of S3, you can use this in the binarystore.xml:

<config version="2">
<chain template="cluster-file-system"/>
</config>

Download Location

This will require access to https://releases.jfrog.io

$ docker pull releases-docker.jfrog.io/jfrog/artifactory-pro:latest

Steps to install

Now that we’ve configured the system.yaml, we can start the first node. For reference, here is what my (basic) complete system.yaml looks like:

shared:
database:
type: postgresql
driver: org.postgresql.Driver
url: jdbc:postgresql://10.150.0.223:5432/artifactory
username: artuser
password: PassJFrog!453
node:
ip: 10.150.0.222
id: nodeone
haEnabled: true
taskAffinity: any

1. Start Artifactory:

$ docker run --name artifactory -v /home/loreny/jfrog/artifactory/var/:/var/opt/jfrog/artifactory -d -p 8081:8081 -p 8082:8082 releases-docker.jfrog.io/jfrog/artifactory-pro:latest

2. Check that artifactory starts up by checking the logs for the following message:

$ docker logs -f artifactory
2021-09-20T20:44:34.610Z [jfrou] [INFO ] [30a6b3f102dc9b52] [local_topology.go:270 ] [main ] -
###############################################################
### All services started successfully in 86.544 seconds ###
###############################################################
3. Once Artifactory comes up, the UI should be accessible at port 8082. Check that Artifactory is in HA mode by running the following REST API and look in the addon array for “ha”:$ curl localhost:8082/artifactory/api/system/version -u admin:password
{
"version" : "7.25.7",
"revision" : "72507900",
"addons" : [ "ha",...

4. For a new node to join a cluster, the nodes must connect to the same database and have the same Master Key. To install additional nodes, you’ll want to use the same system.yaml (change the node IP and ID) and binarystore.xml. The install process is identical, with the exception of needing to copy over a master.key. If you would like to generate your own key ahead of time, you can follow the guide here to do so. This key will be generated for you on first start up of the first node. Copy the master.key from the first node (artifactory/var/etc/security/master.key) to the additional nodes’, at the same file system location. Then you can start the additional nodes with the same docker run command. Note that you will need a license per Artifactory node.

SSL

If HTTPS is required, you can setup a reverse proxy in front of Artifactory. A reverse proxy configuration can be generated in the Artifactory UI by going to Administration->Artifactory->HTTP Settings. We will be using NGINX for this example. This will need to be copied to your NGINX config. You will need to have your own SSL certs and key and place them in the correct directory specified in the NGINX config. Below is a sample configuration for reference. This configuration below assumes that you are installing NGINX on the same machine as Artifactory; please adjust the host as necessary. 

###########################################################
## this configuration was generated by JFrog Artifactory ##
###########################################################

## add ssl entries when https has been set in config
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_certificate /etc/ssl/private/server.key;
ssl_certificate_key /etc/ssl/private/server.crt;
ssl_session_cache shared:SSL:1m;
ssl_prefer_server_ciphers on;
## server configuration
server {
listen 443 ssl;
listen 80 ;
server_name ~(?<repo>.+)\.artifactory_host artifactory_host;

if ($http_x_forwarded_proto = '') {
set $http_x_forwarded_proto $scheme;
}
## Application specific logs
## access_log /var/log/nginx/artifactory_host-access.log timing;
## error_log /var/log/nginx/artifactory_host-error.log;
rewrite ^/$ /ui/ redirect;
rewrite ^/ui$ /ui/ redirect;
rewrite ^/(v1|v2)/(.*) /artifactory/api/docker/$repo/$1/$2;
chunked_transfer_encoding on;
client_max_body_size 0;
location / {
proxy_read_timeout 2400s;
proxy_pass_header Server;
proxy_cookie_path ~*^/.* /;
proxy_buffer_size 128k;
proxy_buffers 40 128k;
proxy_busy_buffers_size 128k;
proxy_pass http://localhost:8082;
proxy_set_header X-JFrog-Override-Base-Url $http_x_forwarded_proto://$host:$server_port;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

location ~ ^/artifactory/ {
proxy_pass http://localhost:8081;
}
}
}

Reverse proxies can also be configured to handle load balancing traffic between nodes. In NGINX’s case, you can add this snippet at the top:

upstream artifactory {
server 10.150.0.222:8082;
server <additional node IP>:8082;
}
upstream artifactory-direct {
server 10.150.0.222:8081;
server <additional node IP>:8081;
}

Then, modify the proxy_pass lines to be:

proxy_pass http://localhost:8082; → proxy_pass http://artifactory;
proxy_pass http://localhost:8081; → proxy_pass http://artifactory-direct;

Otherwise, you may use a dedicated load balancer to handle traffic balancing. 

Note that to support docker requests, you’ll need a reverse proxy or load balancer to handle request rewrites. Also, if you are planning on having a load balancer terminating SSL, and a reverse proxy, you’ll need the below headers to be hard coded to the details of your load balancer:
 

For NGINX:

proxy_set_header X-JFrog-Override-Base-Url https://<LBHOST>:<LBPORT>;
proxy_set_header X-Forwarded-Port <LBPORT>
proxy_set_header X-Forwarded-Proto https

Steps to upgrade

1. Stop and remove the Artifactory container using native Docker commands.
For example:docker stop artifactory
docker rm -f artifactory
2. Set the path to your current Artifactory home, and make sure the directory permissions are set correctly.export ARTIFACTORY_HOME=<Mount Path to your current Artifactory data directory>
chown -R 1030:1030 $ARTIFACTORY_HOME
3. Start the Artifactory containerdocker run --name artifactory -v $ARTIFACTORY_HOME:/var/opt/jfrog/artifactory -d -p 8081:8081 -p 8082:8082 docker.bintray.io/jfrog/artifactory-<pro|oss|cpp-ce>:latest4. Repeat for other nodes.

Tuning Artifactory (Optional)

We have the following optional tuning section to optimize Artifactory for heavier loads – it is a good idea to keep these parameters in mind as your Artifactory instance takes on more load. 

How do I tune Artifactory for heavy loads?

1. javaOpts (heap size) in system.yaml – we recommend at least setting this one:shared:
extraJavaOpts: "-Xms512m -Xmx4g"
2. Customize database connections in system.yaml: artifactory:
database:
maxOpenConnections: 200

access:
database:
maxOpenConnections: 200

metadata:
database:
maxOpenConnections: 200

As a rule of thumb, we require (upto) a number of DB connections based on the following formula:
Total number of connections = (number of nodes) * ((artifactory.database.maxOpenConnections * 2) + access.database.maxOpenConnections + metadata.database.maxOpenConnections) + 50

3. Tune Tomcat threads in system.yaml:

artifactory:
tomcat:
connector:
maxThreads: 400

access:
tomcat:
connector:
maxThreads: 100

When modifying the Access maxThreads, it is required to update the $JFROG_HOME/artifactory/var/etc/artifactory/artifactory.system.properties file with:

artifactory.access.client.max.connections = <VALUE>

4. Tune Async thread pool in the same file. Note the corePoolSize should not be more than 8x the number of CPU cores:
 

artifactory.async.corePoolSize = 32
artifactory.async.poolMaxQueueSize = 100000

Glossary

– System Requirements: https://www.jfrog.com/confluence/display/JFROG/System+Requirements
– External DB + Supported DBs: https://www.jfrog.com/confluence/display/JFROG/Configuring+the+Database
– Postgresql DB: https://www.jfrog.com/confluence/display/JFROG/PostgreSQL
– Filestore: https://www.jfrog.com/confluence/display/JFROG/Configuring+the+Filestore
– HTTPS Settings: https://www.jfrog.com/confluence/display/JFROG/HTTP+Settings
– Nginx install: https://www.nginx.com/resources/wiki/start/topics/tutorials/install/
– JFrog Download: https://jfrog.com/download-jfrog-platform/
– S3 Cluster Binary Provider: https://www.jfrog.com/confluence/display/JFROG/Configuring+the+Filestore#ConfiguringtheFilestore-S3ClusterBinaryProvider
– General Cluster Binary Provider:
https://www.jfrog.com/confluence/display/JFROG/Configuring+the+Filestore#ConfiguringtheFilestore-ConfiguringShardingforHACluster
– Docker Installation: https://www.jfrog.com/confluence/display/JFROG/Installing+Artifactory#InstallingArtifactory-DockerInstallation
https://www.jfrog.com/confluence/display/JFROG/Installing+Artifactory#InstallingArtifactory-LinuxArchive/Docker/RPM/Debian/WindowsHAInstallation
– Docker Upgrade:
https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory#UpgradingArtifactory-DockerUpgradedockerupgr7
– HA Upgrade:
https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory#UpgradingArtifactory-UpgradeSteps.1