In order to increase stability of write-locking method between nodes of your Artifactory Enterprise cluster, we recommend disabling Hazelcast. In addition, you may also disable hazelcast for UI session sharing between nodes.
Artifactory versions 5.5.0 and above to below 6.0.0
Artifactory has been using Hazelcast mainly for locking during write operations and sharing UI sessions between nodes. For each function, you may use following directions to replace the Hazelcast with new method for reasons described below.
I. Write Lock
Instead of using Hazelcast, you can use database for taking care of the write locks. This new method using database provides greater stability over hazelcast. Depending on number of nodes, you may also see increase in performance around write lock because Hazelcast does not need to notify all the nodes for the changes. Instead, a node just need to directly work with database.
- that we recommend a short down time while switching to db lock. The reason we recommend downtime is to avoid scenarios that some of the nodes will use the new locking mechanism and some of the nodes will use the old one. In such case the cluster will break into two groups and each group might interfere each other or corrupt the data. For example, one node will upload binaries to a/b/bin.txt, while the other node might delete the directions a/b. Without full cluster lock we might get into situations where the the file exists without the directories.
- The new database locking mechanism adds its own connection pool (defaults to the value of the pool.max.active value). However, you may need to adjust your database connection limit to accept more connections. For example, if your database is set to accept up to 100 connections from each node, you may consider increasing the limit to 130 concurrent connections per-node, to accommodate the full utilization of the locking connection pool. Your database should accept the number of configured connections per-node multiplied by the number of the nodes in the cluster.
- DB Lock is default in Artifactory 6.0.0 and above (including upgrade)
II. Session Sharing
Session sharing can be made local, with the tradeoff of having to enable sticky sessions on the Load Balander. With this change, all propagation of internal massages between nodes has been refactored and is done via REST, which will allow you to put this mechanism behind an SSL-backed web server and avoid unencrypted communication over TCP if SSL/TLS is enabled on each node or at the load balancer.
I. Hazelcast-replacement for locking provider
Starting Artifactory 5.5, locks can be done by the database instead of the Hazelcast. You may make the change by using the steps below
1. Shut down all the nodes (See above Details to see why)
2. (Optional) increase DB connections both in Artifactory (pool.max.idle) and in Database. The number of maximum DB connections combined for all the nodes should be less than DB's maximum. So, [pool.max.active] * [number of nodes] < [Max DB connections]
For more information on the database monitoring and tuning, please refer to this article (monitoring-and-optimizing-artifactory-performance/) and an open source monitoring tool.
3. Set following property at artifactory.system.properties of each node
4. Start each node, starting with primary node
II. Hazelcast-replacement for session sharing
Use following steps to remove the shared session (and Hazelcast):
2. Set following property at artifactory.system.properties
3. Restart each node