ARTIFACTORY: Configuring Filestore for High Availability with Emphasis on Redundancy and Lenient Limit

Ensuring a robust and highly available filestore configuration is crucial for seamless operations and optimal performance of your system. In this article, we will explore the essential aspects of configuring Filestore for High Availability (HA) and focus on two critical elements: "Redundancy" and "Lenient Limit."
 

High Availability Filestore Configuration:

When setting up Filestore for High Availability, it is imperative to have a resilient storage solution that safeguards against data loss and minimizes downtime. The HA configuration is designed to ensure data accessibility, data integrity, and continued service even in the face of hardware failures or other unforeseen issues.
 

Redundancy 

Redundancy is a fundamental concept in HA Filestore configuration. It refers to the duplication of data across multiple physical devices or nodes within the storage system. By distributing data redundantly, the filestore gains resilience against failures since it can continue to function seamlessly even if some nodes experience issues.

In a redundant Filestore configuration, data is mirrored or replicated across multiple nodes. This means that If the filestore of the node where the artifact is located becomes unavailable, then Artifactory can still serve the artifact because it was redundantly saved in another shard. This mechanism ensures uninterrupted access to files and minimizes the risk of data loss, thereby providing a reliable and stable system.
 

Lenient Limit

The "Lenient Limit" configuration parameter plays a significant role in balancing the performance and capacity of the Filestore system. It establishes a threshold that determines when the system can handle exceeding the specified storage limit. The Lenient Limit represents the minimum number of successful writes required to validate a "WRITE" transaction or upload process. By default, its value is set to 1. If you set it to 0, Artifactory will consider it equivalent to the redundancy parameter. The WRITE transaction will be confirmed once the number of binary copies meets the criteria defined by either the "Lenient Limit" or the "Redundancy." To enhance the transfer process, the Write transaction relies on concurrent streams to duplicate the binaries across multiple shards. This ensures that a minimum number of successful writes must be maintained for an upload to be deemed successful.  During the subsequent balance cycle, triggered by the Garbage Collection (GC) mechanism, the binary will be distributed to a sufficient number of nodes to preserve redundancy commitments.
 

How the Mechanism Works (examples)

Artifactory with 3 nodes in HA cluster with Filestore.

Set Redundancy = 3,  LenientLimit = 1

• If node N1 & N3 goes down, the upload of the files would still take place on N2. Once the next balance cycle (triggered with the GC mechanism) happens it will eventually transfer the binary to the rest of the active nodes (N1 & N3), maintaining the redundancy of the data across all the active nodes. 

Set Redundancy = 3,  LenientLimit = 2

• If node N1 & N3 goes down, the lenient limit value will be set greater than the number of active nodes making the writing stop. The download of the files will not get impacted.

The redundancy parameter would ensure that the number of copies of the binary will be stored in all the filestore locations of the defined number of nodes in Artifactory. The value of redundancy should be equal to or less than the number of filestore mounts in the Artifactory. The lenient limit would ensure the minimum number of successful writes that must be maintained for an upload to be successful. 



For more information of the various filestore configurations to select from, depending on the specific needs of your organization, please refer to our Artifactory Filestore.