ARTIFACTORY: How to calculate the total storage usage of different repository types using Storage Summary from Artifactory

Introduction 

To calculate the total storage usage for repositories of different types (e.g., CACHE, LOCAL) directly from Artifactory storage summary, you can first use Artifactory storage summary API or storage-summary.json found within the Artifactory support bundle to retrieve the storage summary in JSON format. 
Once you have this JSON data, you can use jq (lightweight and flexible command-line JSON processor) to filter and sum the usedSpaceInBytes for different repository types.


Step-by-Step Guide:

Fetching the Storage Summary from Artifactory:
Use the following curl command to fetch the storage summary from Artifactory instance:

curl -X GET -u admin:password http://localhost:8081/artifactory/api/storageinfo

• Replace admin:password with your Artifactory credentials.
  
• This command will retrieve the storage summary in JSON format. The response will contain details about all repositories, including the type, space usage etc.

Example Response:

{
  "binariesSummary" : {
    "binariesCount" : "118,202",
    "binariesSize" : "3.18 TB",
    "artifactsSize" : "3.48 TB",
    "optimization" : "91.5%",
    "itemsCount" : "156,763",
    "artifactsCount" : "127,801"
  },
  "fileStoreSummary" : {
    "storageType" : "user-chain",
    "storageDirectory" : "/opt/jfrog/artifactory/var/data/artifactory/cache",
    "totalSpace" : "491.96 GB",
    "usedSpace" : "4.87 GB (0.99%)",
    "freeSpace" : "487.09 GB (99.01%)"
  },
  "repositoriesSummaryList" : [ {
    "repoKey" : "terraform-provider-virtual",
    "repoType" : "VIRTUAL",
    "foldersCount" : 0,
    "filesCount" : 0,
    "usedSpace" : "0 bytes",
    "usedSpaceInBytes" : 0,
    "itemsCount" : 0,
    "packageType" : "Terraform",
    "projectKey" : "default",
    "percentage" : "0%"
  },
  {
    "repoKey" : "docker-test-local",
    "repoType" : "LOCAL",
    "foldersCount" : 0,
    "filesCount" : 0,
    "usedSpace" : "0 bytes",
    "usedSpaceInBytes" : 2000000000,
    "itemsCount" : 0,
    "packageType" : "Docker",
    "projectKey" : "default",
    "percentage" : "0%"
  },
  {
    "repoKey" : "maven-remote-cache",
    "repoType" : "CACHE",
    "foldersCount" : 0,
    "filesCount" : 0,
    "usedSpace" : "0 bytes",
    "usedSpaceInBytes" : 3000000000,
    "itemsCount" : 0,
    "packageType" : "Maven",
    "projectKey" : "default",
    "percentage" : "0%"
  }]
}

Note

Alternatively, The storage-summary.json file can be found within the Artifactory support bundle. This file contains similar information to the API response and can be processed in the same way.


Extracting and Summing usedSpaceInBytes for a Specific repoType Using jq:
Once the storage summary is fetched, you can pipe the output into jq to calculate the total used space in bytes format for a specific repoType.

Example: Get Total Used Space for CACHE Repositories

To get the total used space for repositories of type CACHE, run the following command:

curl -X GET -u admin:password http://localhost:8081/artifactory/api/storageinfo | jq '[.repositoriesSummaryList[] | select(.repoType == "CACHE") | {usedSpaceInBytes: .usedSpaceInBytes}] | map(.usedSpaceInBytes) | add'

Explanation:

• curl -X GET ...: Fetches the storage summary in JSON format.
  
• jq '[.repositoriesSummaryList[] ... ]': Processes the JSON to extract the usedSpaceInBytes for repositories of type CACHE.
  
• map(.usedSpaceInBytes): Converts the values into an array.
  
• add: Sums the values in the array, giving you the total used space for all CACHE repositories.

Example Output:
3819997769406

Calculating Used Space for Other repoType Values:
You can similarly calculate the total space for other repository types. For example, to get the total used space for LOCAL repositories:

curl -X GET -u admin:password http://localhost:8081/artifactory/api/storageinfo | jq '[.repositoriesSummaryList[] | select(.repoType == "LOCAL") | {usedSpaceInBytes: .usedSpaceInBytes}] | map(.usedSpaceInBytes) | add'

 

Example Output:
2427057871


Comparing Storage Across Different Artifactory Clusters:

If you have multiple Artifactory clusters, you can perform the same process for each cluster to compare storage usage across them.
For example, to compare the CACHE storage between two clusters (cluster1 and cluster2), you would run:

Cluster 1:

curl -X GET -u admin:password http://cluster1:8081/artifactory/api/storageinfo | jq '[.repositoriesSummaryList[] | select(.repoType == "CACHE") | {usedSpaceInBytes: .usedSpaceInBytes}] | map(.usedSpaceInBytes) | add'

Cluster 2:

curl -X GET -u admin:password http://cluster2:8081/artifactory/api/storageinfo | jq '[.repositoriesSummaryList[] | select(.repoType == "CACHE") | {usedSpaceInBytes: .usedSpaceInBytes}] | map(.usedSpaceInBytes) | add'

This will allow you to compare the total used space for CACHE repositories between the two clusters.

Conclusion

This approach provides an efficient way to monitor and compare storage usage across different repository types and Artifactory clusters.