Filestore Sharding

As companies scale up and grow into enterprises, so does the scale of their filestores. As filestores grow to terabytes and beyond, enterprises face challenges of performance, reliability, maintenance and more. To meet the challenges of enterprise scale filestores, Artifactory introduces filestore sharding.

A sharded filestore is one that is implemented on a number of physical mounts (M) which store binary objects with redundancy (R), where R <= M. For example, the diagram below represents a sharded filestore where M=3 and R=2. In other words, the filestore consists of 3 physical mounts which store each binary in two copies.

Sharding overcomes the challenges of scaling enterprise filestores.

Unmatched stability and reliability

Thanks to redundant storage of binaries, the system can withstand any mount going down as long as the remaining mounts can support the redundancy configured (i.e., M >= R).

Unlimited scalability

If the underlying storage available approaches depletion, you only need to add another mount; a process that requires no downtime of the filestore. Once the mount is up and running, the system invokes balancing mechanisms to regenerate the filestore redundancy according to configuration parameters you control.

Filestore performance optimization

Artifactory’s filestore sharding implementation offers several configuration parameters that allow you to optimize how binaries are read from or written to the filestore according to your specific system’s requirements. As binaries are uploaded or deleted, the sharding mechanism invokes balancing processes to make sure that binaries are uniformly distributed among the filestore mounts according to the redundancy configured in the system.

One of the growing pains enterprises experience when scaling up is the challenge of keeping their products and services fully functional, reliable and performant. Exponentially growing filestores must eventually be migrated to a model that can sustain growth and keep the enterprise systems working optimally. Artifactory’s filestore sharding provides a flexible solution that lets enterprises grow their data at any pace with zero downtime while providing the tools to tweak filestore parameters for optimal performance.