Eventual Binary Provider

JFrog Installation & Setup Documentation

Content Type
Installation & Setup
ft:sourceType
Paligo

This binary provider is not independent and will always be used as part of a template chain for a remote filestore that may exhibit upload latency (for example S3 or GCS).

Eventual Binary Provider Configuration

To overcome potential latency, files are first written to a folder called “eventual” under the baseDataDir in local storage, and then later uploaded to persistent storage with the cloud provider. The default location of the eventual folder is under the $JFROG_HOME/artifactory/data/artifactory folder and is not configurable. You need to make sure that Artifactory has full read/write permissions to this location.

There are three additional folders under the eventual folder:

  • _pre: part of the persistence mechanism that ensures all files are valid before being uploaded to the remote filestore

  • _add: handles upload of files to the remote filestore

  • _delete: handles deletion of files from the remote filestore

Eventual Binary Provider Example

The following example shows a configuration that uses S3 for persistent storage after temporary storage with an eventual binary provider. The eventual provider configures 10 parallel threads for uploading and a lock timeout of 180 seconds.

<config version="2">
   <chain template="s3-storage-v3"/>
   <!-- The S3 binary provider configuration -->
   <provider id="s3-storage-v3" type="s3-storage-v3">
       <endpoint>s3.amazonaws.com</endpoint>
       <bucketName>bucketName</bucketName>
       <path>pathPrefix</path>
       <region>s3Region</region>
       <identity>yourIdentity</identity>
       <credential>yourCredentials</credential>
   </provider>

   <!-- The eventual provider configuration -->
   <provider id="eventual" type="eventual">
        <numberOfThreads>10</numberOfThreads>       
        <timeout>180000</timeout>
    </provider>
</config>

Where:

Parameter

Description

type

eventual

timeout

The maximum amount of time a file may be locked while it is being written to or deleted from the filesystem.

dispatcherInterval

Default: 5000 ms

The interval between which the provider scans the “eventual” folder to check for files that should be uploaded to persistent storage.

numberOfThreads

Default: 5

The number of parallel threads that should be allocated for uploading files to persistent storage.

passThrough

Default: false

When this flag is set to true, the eventual upload mechanism functions like the direct upload mechanism. All new binaries will be saved directly on the remote provider and the eventual mechanism is bypassed completely. This can be used to aid migration from eventual to direct providers. For more info, click here.

Warning

This parameter should not be used on Artifactory versions between 7.59 and 7.63.11.