Optimize Cold Storage Performance

JFrog Platform Administration Documentation

Content Type
Administration / Platform
ft:sourceType
Paligo

Cold Storage includes several properties that can be used to configure and optimize its performance.

Warning

The list of parameters and their corresponding values that are shown below are for reference only. It is highly recommended that you do not change these values.

Live Artifactory Properties

The following properties can be configured in the Live Artifactory's $JFROG_HOME/etc/artifactory/artifactory.system.properties file.

Parameter

Description

Range/ Default

Basic Properties

artifactory.retention.warm.archive.concurrency.maxLevel

Number of threads that will be used per policy to archive items.

Default: 5

artifactory.retention.warm.restore.concurrency.maxLevel

Number of threads that will be used to restore items.

Default: 5

Advanced Properties

artifactory.retention.cold.http.client.max.total.connections

Number of HTTP connections that are allowed to deploy artifacts from the Live instance to the Cold instance.

Default: 50

artifactory.retention.cold.http.client.socketTimeout

Maximum socket timeout for an HTTP connection.

Default: 15,000

artifactory.retention.cold.http.client.max.connections.per.route

Number of HTTP peers per route that are allowed to deploy artifacts from the Live instance to the Cold instance.

Default: 50

artifactory.retention.warm.restore.pull.items.intervalSecs

Setting for how often the Live instance will try to pull restored items from the Cold instance.

Default: 120

Recommended: 1,800

artifactory.retention.warm.restore.batchSize

Setting for how many restored items will be pulled from the Cold instance to the Live instance.

Default: 1,000

artifactory.retention.warm.restore.bandwidth.maxSizeMb

Approximate maximum allowed bandwidth for all restore operations.

Default: 0

artifactory.retention.warm.restore.error.threshold.percent

Error threshold to stop restore operation.

Default: 10

artifactory.retention.warm.archive.batchSize

Setting for how many items are fetched from the database during archive policy execution.

Default: 10,000

artifactory.retention.warm.archive.bandwidth.maxSizeMb

Approximate max allowed bandwidth for a single archive policy execution.

Default: 0

artifactory.retention.warm.archive.error.threshold.percent

Stop execution if there are 10% errors during an archive process.

Default: 10

retentionWarmPackageArchiveQueryLimit

Specifies the maximum amount of packages archived per one batch.

Default: 10,000

retentionColdHttpClientRetryCount

Specifies the maximum number of retries in the event of Cold Server communication failure.

Default: 30

retentionColdHttpClientRetryDelaySecs

Specifies the delay (in seconds) between retries if there is a Cold Server communication failure.

Default: 2

Cold Artifactory Properties

The following properties can be configured in the Cold Artifactory System YAML.Artifactory System YAML

Parameter

Description

Range/ Default

Advanced Properties

artifactory.retention.cold.restore.render.tree.concurrency.level

Number of threads that will be used when processing the restore request.

Default: 5

artifactory.retention.cold.restore.status.mapper.intervalSecs

By default, on the Cold Artifactory instance, the cron job rate to check the restore status for each archived item is 120 seconds (2min). This can be reconfigured for cost optimization.

Default: 120

Recommended: 10800

artifactory.retention.cold.restore.cleanup.intervalSecs

Interval for the cleanup job.

Default: 300

The following property can be configured in the Cold Artifactory's $JFROG_HOME/etc/artifactory/artifactory.system.properties file.

Parameter

Description

Range/ Default

retention.warm.restore.artifact.limit

The default maximum number of artifacts that can be restored during a single run is 30k. This value can be configured higher or lower using the retention.warm.restore.artifact.limit property. During the restore process, if the number of artifacts crosses the max limit, the restore process terminates after completing the restoration of artifacts to the maximum limit and the cause for termination is logged.

Default: 30k