XRAY: Xray retention period / Why is my repository not fully indexed?

XRAY: Xray retention period / Why is my repository not fully indexed?

Products
JFrog_Xray
Content Type
User_Guide
AuthorFullName__c
Reut Cohen, Ori Zohar
articleNumber
000005714
FirstPublishedDate
2023-05-09T13:50:46Z
lastModifiedDate
2023-05-09
VersionNumber
6
Introduction 
Under Administration → Xray Settings → Index Resources, you can view the Index Status of your repositories.

In some cases, you may notice that a repository is not fully indexed (i.e., it does not show 100%). This commonly occurs due to one of the following reasons:
  1. Expired data due to retention policy
     Xray retention period specifies how long artifact scan data is kept. Prior to Xray 3.41.4, retention was automatically set to "forever."
    Currently, if no retention period is explicitly set, the default is 90 days for repositories and 15 days for builds. Once the retention period expires, all Xray scan data related to that repository or build is deleted from the Xray database.
    
  2. Artifacts existed prior to indexing
    If artifacts were already present in the repository before indexing was enabled, existing artifacts will not be indexed automatically.
    By default Xray will only scan new artifacts added after the index is enabled.
In both scenarios, the solution is the same: manually reindex the repository to ensure all artifacts are included.


Resolution 
How to recover expired or unindexed data

To recover missing data, reindex the repository:
  1. Navigate to Xray → Settings → Index Resources.
    
  2. Select ‘Index Now’ and then ‘Index All Artifacts’.
     
    User-added image 
    
    
For builds, all Xray data is deleted after 15 days. To recover build data, republish or rescan the build using either:
For more information, please refer to: 
How to index and scan existing builds and artifacts

How to extend or change the retention period

You can increase the retention period to prevent future data loss:
 
  1. Navigate to Xray → Settings → Index Resources → Configure.
    
  2. Specify the desired number of days to keep Xray data for that repository (up to 1,000 days).
     
    User-added image 
    
    
The retention period could be configured for up to 1,000 days.

For more information, see the Xray Index Resources / Retention Period documentation.

Important
It is important to point out that reindexing an entire repository (or multiple repositories) is a heavy operation, depending on the number of artifacts in the repository. Therefore, it is recommended to perform this in smaller chunks, using the date filter or the API: Scan Now API.
Additionally, it is better to reindex repo by repo rather than attempting to reindex all not-fully-indexed repositories at once. Conclusion   Repositories may not be fully indexed either due to expired retention data or because artifacts existed before indexing was enabled. Reindexing the repository and adjusting the retention period ensures all artifacts are scanned and future data is retained, keeping Xray scans complete and up to date.