Indexing Xray Resources

JFrog Security Documentation

ft:sourceType
Paligo

Overview

To avoid lengthy and intensive analysis processes, Xray does not automatically analyze all the resources in the system but allows you to manually select the repositories, builds, and release bundles to be indexed.

Add Resources for Indexing

To configure your indexed resources, in the Administration module, go to Xray | Settings | General and click Indexed Resources.

From the Indexed Resources page do the following:

To add resources for indexing:

  1. Select the resource type to index (Repositories | Builds | Release Bundles| Release Bundle V2).

  2. Add Repositories/Builds/Release Bundles from the available resources in Artifactory.

    • For repositories, from the list of available repositories in Artifactory

    • For builds and release Bundles, according to Name or Pattern

  3. Review the list of added resources.

To remove indexed resources:

  1. From the Resources tab, select Add a Resource.

  2. In the list of resources in the right column, select the resources you want to remove from being indexed and click the arrow to move them to the right column.

    This will remove the resources from the list and Xray will stop indexing those resources.

Indexing All Specific Resources

To index all current and any future specific resources, you can add an Include Pattern containing */**

Note

Index Artifacts History and the Xray Data Retention features require Artifactory version 7.33.x and above.

Index Artifacts History

Index Artifacts history enables you to narrow down the selection criteria of artifacts in the repositories you want to index. Because indexing entire repositories that are large and contain a large number of artifacts all at once may be too lengthy of a process, and the data consumption that is required will consume a large amount of disk space, the index artifacts history feature offers a solution to this issue.

You can select which artifacts to index in a repository by doing one or a combination of the following:

  • Limit which artifacts to index, by selecting a time range for artifacts that were loaded in the last number of days you set.

  • Define a repository path to include or exclude an inner folder within the repository, to index artifacts that reside within that folder.

  • Use a combination of both a time range and a path, for example, set a repository path to a production folder within the repository, and set the time range to only index artifacts within that folder that were uploaded in the last 90 days.

  • Use a combination of both all artifacts and a path, for example, to index all artifacts in the production folder within the repository.

Set a Retention Period

As Xray's indexed resources are retained in the system, this can result in large amounts of data being stored, which impacts your storage and causes performance issues. Starting from Xray version 3.41.4, you can set retention periods for repositories and builds. The retention period defines how long Xray will retain artifacts scan data, and after the set retention period, this data is deleted, thus improving performance and freeing up storage space.

The default period is 90 days. You can change the default period using the server.repo.defaultRetentionDaysForIndexedRepo in the Xray System YAML.Xray System YAML

Repositories Retention Period

To set the retention period for a repository, do the following:

  1. In the Administration module, go to Xray | Settings | General and click Indexed Resources.

    image2022-1-4_15-42-20.png
  2. Select the repository or multiple repositories and select Configure.

  3. Select one of the following:

    • Any artifact from the last number of days: The artifact will be retained for the number of days you set here, after the artifact is scanned. This will apply to all artifacts in the repository.

      image2022-1-4_15-46-27.png
    • By Pattern: By pattern enables you to set a more granular retention period. It enables you to scan future artifacts within the specific path, and set a retention period for the historical data of artifacts after they are scanned.

      • Index: If checked, Xray will scan newly added artifacts in the path. Note that existing artifacts will not be scanned. If the folder contains existing artifacts that have been scanned, and you do not want to index new artifacts in that folder, you can choose not to index that folder.

      • Retention period: The number of days to retain artifacts data after they are scanned by Xray.

        For example, if you have one repository that contains all of your artifacts, you can set a retention period for artifacts within a folder in the repository. Let's assume your repository contains a production folder, and you would like to set a retention period for artifacts within that folder. You can provide the path to that folder as an include pattern and set the retention period. If that folder also contains other folders that you do not want to be scanned or retained, you can exclude it using the exclude pattern.

        Note

        Patterns are set using simple comma-separated wildcard patterns for repository artifact paths (with no leading slash). Ant-style path expressions are supported (*, **, ?). For example: "org/apache/**"

        Patterns are limited to 10 patterns per repository.

        If you select by pattern, you must define a retention period for all other artifacts in the repository in the All Other Artifacts setting.

        image2022-1-4_15-47-57.png
Builds Retention Period

To set the retention period for a Build, do the following:

  1. In the Administration module, go to Xray | Settings | General and click Indexed Resources. Select the Builds tab.

  2. Click the Set Retention icon next to the Build you want to configure.

    image2022-1-5_13-21-10.png
  3. Set the retention period. The default is 15 days.

    image2022-1-5_13-22-9.png
Xray Data Retention REST API Support

Repository Configuration is also supported through the following REST APIs:

Xray Data Retention System YAML Configurations

The following table lists all the configuration system parameters that support Xray Data Retention:

System Parameter

Description

Default Value

server.repo.maxRetentionDaysLimit

Max limit of retention period for artifacts in a repository.

1000

server.repo.maxRepoPathsPatternsAllowed

Max limit of number of patterns in repository paths per repository.

10

server.repo.defaultRetentionDaysForIndexedRepo

Default retention period for artifacts of an index repository. This value is applied when the retention period is not configured.

90

server.repo.defaultRetentionDaysForNonIndexedRepo

Default retention period for artifacts in a non-indexed repository.

3

server.disableXrayDataCleanupJob

Disable data cleanup.

FALSE

server.build.defaultRetentionDaysForIndexedBuild

Default retention period for indexed builds. This value is applied when the retention period is not configured for the build name or pattern.

15

server.build.defaultRetentionDaysForNonIndexedBuild

Default retention period for builds that are not marked for indexing.

3