Pipeline Source Sync Recovery

JFrog Pipelines Documentation

Products
JFrog Pipelines
Content Type
User Guide
ft:sourceType
Paligo

Pipelines provides an option to enable auto-sync for pipelines sources that are out of sync. Pipeline sources and related objects can become outdated if the webhooks from SCM providers are not received and if the commit event to a branch is not notified to Pipelines. In addition, problems with a webhook's origin, network, and Pipelines services can also cause pipeline source sync issues. In these scenarios, while the pipeline might look like it is in a valid state, it is actually out of sync.

Pipeline source sync recovery ensures that if a step fails due to the pipeline source not being in sync, all the matching GitRepo resources using the same repo path, integration and branch are automatically re-synced, prompting an automatic trigger of the step.

Note

This feature does not include recovery of lost PRs, releases and tags events.

Enabling Auto-Sync

The pipeline source auto-sync feature is not available by default. To add and enable this feature, in the Core Services Configurations section in the Pipelines System YAML, add the tag autoSyncResourceIfOutdated and set it as true.Pipelines System YAML

autoSyncResourceIfOutdated: true

Auto-Sync Behavior

This section provides information about the behavior of pipeline source auto-sync feature in various scenarios.

Missing Commits

If the step in a pipeline is triggered as the result of an external event and if its input resources commit is not the latest, then that step fails in the ‘Processing required resources’ phase with the following error message:

Your GitRepo resource is outdated. We will sync the resource to the updated version shortly.

In this scenario, after the step completes, all git repo resources matching the same repo path, integration, and branch are auto-synced and the the failed step is automatically re-triggered (unless the trigger tag is set to false).

Change in Config File

There could be cases where there is a change in the pipelines config file (yaml), but the webhook does not work. In this case, when the pipeline is manually triggered and it fails, the pipeline source, which is using the same branch, repo, and integration, is auto-synced.

Commit Gap

To identify if the pipeline source needs to be synced, Pipelines checks if the latest SHA available for the resource matches the previous SHA of the webhook event. If they do not match, the pipeline source is auto-synced.

Trigger is Set to False

By default, any incoming payload from the webhook will trigger the execution of a step, but this can be changed by setting trigger tag as false. In this case, even after the pipeline source is auto-synced, it will not automatically trigger the step.

In such scenarios, you have to manually trigger the step or set the trigger tag as true.