How do I tune Artifactory for heavy loads?

Ariel Kabov
2020-09-03 07:19

Relevant Versions: Artifactory 7 and above.
A tuning guide for previous versions is available here.

Artifactory comes with a predefined set of default configurations and parameters. The default Artifactory should handle up to ~200 concurrent connections well.
If you believe your Artifactory server is under-utilized, or in order to allow it to handle more processes at a given moment, it is possible to tune Artifactory to support a higher load.
While it is always possible to scale horizontally by adding additional nodes to your HA cluster, here we will focus on a more vertical scale.

Recommendation: The more crucial Artifactory becomes in your organization, the more crucial will be to have a monitoring system to look over Artifactory.
You may read further at Monitoring and Optimizing Artifactory Performance.

JVM Memory

By default, Artifactory comes with a predefined JVM memory limit. 
To modify the JVM memory allocation, please refer to the Product Configuration section that is part of the installation guide. Be advised to follow our hardware recommendations.
When increasing the JVM memory allocation, make sure you leave at least 30% of the total RAM to the OS and other services.

Database connections

We can alter the maximum connections an Artifactory node can open to the DB by modifying the Artifactory System YAML.

Default values:artifactory:
  database:
    maxOpenConnections: 100
...    
access:
  database:
    maxOpenConnections: 100
...
metadata:
  database:
    maxOpenConnections: 100

Tuning example:artifactory:
  database:
    maxOpenConnections: 300
...    
access:
  database:
    maxOpenConnections: 300
...
metadata:
  database:
    maxOpenConnections: 300

Important: The Artifactory maxOpenConnections parameter is being used also by the Artifactory Session Management mechanism.
This means once the above example is used, the Artifactory node will open up to 1200 DB connections. 
Therefore we need to make sure the DB can accommodate the total number of connections all Artifactory nodes can open.
As a rule of thumb we will require from the DB a number of connections based on:Total # of connections = (number of nodes) * ((artifactory.database.maxOpenConnections * 2) + access.database.maxOpenConnections + metadata.database.maxOpenConnections) + 50;

*The extra 50 connections are to provide extra breathing room in situations where all DB connection pools are exhausted.

Tomcat HTTP Connections / Threads

Artifactory runs on top of Apache Tomcat, which manages the incoming HTTP connection pools.
This sets the number of concurrent HTTP connections Artifactory can serve.
We can override the default thread pool limit by modifying the Artifactory System YAML.

Default values:artifactory:
  tomcat:
    connector:
      maxThreads: 200
...
access:
  tomcat:
    connector:
      maxThreads: 50

Tuning example:artifactory:
  tomcat:
    connector:
      maxThreads: 600
...
access:
  tomcat:
    connector:
      maxThreads: 150

Important: When modifying the Access maxThreads, it is required to update the $JFROG_HOME/artifactory/var/etc/artifactory/artifactory.system.properties file with:artifactory.access.client.max.connections = <VALUE>
This is to modify the internal HTTP connection pool Artifactory uses to internally interact with Access. 

Artifactory async Thread Pool

One of the most important thread pools in Artifactory is the “async” thread pool. This one defines the number of processes that can run in parallel.
In addition to configuring the total number of parallel processes, we can also modify the maximum number of processes that can be queued.
This is configured in $JFROG_HOME/artifactory/var/etc/artifactory/artifactory.system.properties.

Default values:
(this means the machines CPU cores times 4)artifactory.async.corePoolSize = (4 * Runtime.getRuntime().availableProcessors()) artifactory.async.poolMaxQueueSize = 10000

Tuning example:
(Shouldn’t be more than 8x the machine CPU cores)artifactory.async.corePoolSize = 128artifactory.async.poolMaxQueueSize = 100000

Garbage Collection

By default, the Artifactory Garbage Collection is configured to run every 4 hours.
The GC is a very resource-consuming operation, and if you see correlations between the running period of the GC to slow performance, we would recommend you to alter the Artifactory GC (not related JVM GC) to run at non-peak hours.

HTTP Client 

Artifactory manages a separate connection pool for outgoing HTTP requests per remote repository.
This connection pool is limited by default to 50 concurrent connections, and up to 50 concurrent connections per unique route.

This is configured in $JFROG_HOME/artifactory/var/etc/artifactory/artifactory.system.properties.

Default values:artifactory.http.client.max.total.connections = 50
artifactory.http.client.max.connections.per.route = 50

Tuning example:artifactory.http.client.max.total.connections = 150
artifactory.http.client.max.connections.per.route = 120

Bypassing the Router

The Artifactory 7 System Architecture provides us a flexible way to modify the flow a request will be processed by the Artifactory services.
You can benefit from an improved performance by bypassing the Router service for API requests to Artifactory.
This can be achieved using a Reverse Proxy such as NGINX or Apache HTTPD.
By having a reverse proxy to redirect requests from $JFROG_URL/artifactory directly to $ARTIFACTORY_NODE:8081/artifactory, you will bypass the Router service.
When under high load this can help to better distribute the requests in advance.

Filestore Configurations

Artifactory supports different backend storage configurations to store the Artifactory filestore.
For scenarios where the configured storage is not local, a performance benefit can be using a large Cache-FS provider mounted locally for each node.
Cached files will be served quickly, and therefore having a large Cache FS provider will be a performance gain.

Some filestore providers allow tuning and modifications of parameters. For instance: 
Eventual: "numberOfThreads".
Eventual-Cluster: "maxWorkers".
Remote: "maxConnections".

Read more at the Best Practices for Managing Your Artifactory Filestore white paper.