Pipelines Metrics

JFrog Platform Administration Documentation

Content Type
Administration / Platform
ft:sourceType
Paligo

The following are the three metric log files created for Pipelines:

  • Open Metrics Format:

    • Pipeline API Metrics:$JF_PRODUCT_HOME/var/log/api-metrics.log

  • Non-Open Metrics Format:

    • Pipeline Reqsealer Event Metrics:$JF_PRODUCT_HOME/var/log/reqsealer-activity-event.log

    • Pipeline Sync Event Metrics: $JF_PRODUCT_HOME/var/log/pipelinesync-activity-event.log

Pipelines Open Metrics Format

The Get Pipelines Metrics Data REST API returns the following metrics in Open Metrics format.Get Pipelines Metrics Data

Metric

Description

sys_cpu_user_seconds

User CPU usage time for thethe pipelineprocess in seconds

sys_cpu_system_seconds

System CPU usage time forthe pipelineprocess in seconds

sys_cpu_total_seconds

Total CPU usage time for the pipeline process in seconds

nodejs_heap_read_only_space_total

Total size allocated for V8 heap segment “read_only_space”

nodejs_heap_read_only_space_used_total

Used size for V8 heap segment “read_only_space”

nodejs_heap_new_space_total

Total size allocated for V8 heap segment “new_space”

nodejs_heap_new_space_used_total

Used size for V8 heap segment “new_space”

nodejs_heap_old_space_total

Total size allocated for V8 heap segment “old_space”

nodejs_heap_old_space_used_total

Used size for V8 heap segment “old_space”

nodejs_heap_code_space_total

Total size allocated for V8 heap segment “code_space”

nodejs_heap_code_space_used_total

Used size for V8 heap segment “code_space”

nodejs_heap_map_space_total

Total size allocated for V8 heap segment “max_space”

nodejs_heap_map_space_used_total

Used size for V8 heap segment “max_space”

nodejs_heap_large_object_space_total

Total size allocated for V8 heap segment “large_object_space”

nodejs_heap_large_object_space_used_total

Used size for V8 heap segment “large_object_space”

nodejs_heap_code_large_object_space_total

Total size allocated for V8 heap segment “code_large_object_space”

nodejs_heap_code_large_object_space_used_total

Used size for V8 heap segment “code_large_object_space”

nodejs_heap_new_large_object_space_total

Total size allocated for V8 heap segment “new_large_object_space”

nodejs_heap_new_large_object_space_used_total

Used size for V8 heap segment “new_large_object_space”

sys_memory_free_bytes

Host free virtual memory

sys_memory_total_bytes

Host total virtual memory

jfpip_pipelines_per_project_count

Note

In Pipelines 1.24 and prior, this is called jfpip_pipelines_per_project_count_count.

Number of Pipelines Per Project

jfpip_pipelines_count

Note

In Pipelines 1.24 and prior, this is called jfpip_pipelines_count_count.

Number of Total Pipelines

jfpip_queue_messages_total_count

Messages Count for the Queue

jfpip_nodepool_provisionstatus_success_count

Number of node with SUCCESS provisioned status

jfpip_nodepool_provisionstatus_cached_count

Number of node with CACHED provisioned status

jfpip_nodepool_provisionstatus_processing_count

Number of node with PROCESSING provisioned status

jfpip_nodepool_provisionstatus_failure_count

Number of node with FAILURE provisioned status

jfpip_nodepool_provisionstatus_waiting_count

Number of node with WAITING provisioned status

jfpip_concurrent_active_builds_count

Active Concurrent Build Count

jfpip_concurrent_allowed_builds_count

Allowed Concurrent Build Count

jfpip_concurrent_available_builds_count

Available Concurrent Build Count

Note

All node.js heap size statistics are captured using the v8.getHeapSpaceStatistics()API.

Pipeline Logs

The api-metrics.logwill contain system metrics such as:

  • Total disk space used

  • Total disk space free

  • Time CPU is used by the process

  • Node JS Heap related information

Non-Open Metrics Format

In addition to the metrics mentioned about, Pipelines supports the following custom activity-based Event Metrics:

  • Pipeline Run & Step Events: For every pipeline run, two types of metrics can be found in reqsealer-activity-event.log. One entry for each step status and one entry for overall pipeline status.

    {"timestamp":"2022-04-05T08:30:10.088Z","startedAt":"2022-04-05T08:30:03.986Z","queuedAt":"2022-04-05T08:30:03.010Z","domain":"step","pipelineName":"my_pipeline_2","triggeredBy":"admin","branchName":"master","stepName":"p2_s1","runNumber":2,"status":"success","durationMillis":6102,"outputArtifactsCount":0,"outputResourcesCount":0}
    {"timestamp":"2022-04-05T08:30:10.088Z","startedAt":"2022-04-05T08:30:03.986Z","domain":"run","pipelineName":"my_pipeline_2","triggeredBy":"admin","branchName":"master","runNumber":2,"status":"success","durationMillis":6102}
  • Pipeline Sync Events:For every pipeline sync activity, the following metrics can be found in pipelinesync-activity-event.log.

    {"timestamp":"2022-04-06T10:00:45.673Z","domain":"pipelineSync","pipelineSourceName":"Sample","repositoryName":"a-0908/myFirstRepo","branch":"master","status":"success","durationMillis":10498}
  • Webhook Events (Pipelines 1.25 and above):For every pipeline sync activity, the following metrics can be found in pipelinesync-activity-event.log.

    {"timestamp":"2022-06-10T16:29:29.894Z","domain":"webhook","status":"success","durationMillis":533,"webhookId":"11819184-2d88-4180-92da-aa13092d0ca4","integration":"my_bitbucket","source":"gitrepo","eventType":"branchCreated","branchName":"kt4","repositoryName":"krishnakadiyam/jfrog-pipelines-second"}
    {"timestamp":"2022-06-10T16:29:40.845Z","domain":"webhook","status":"success","durationMillis":323,"webhookId":"6d098e3a-7b4b-427c-ba53-b1174baeeabd","integration":"my_bitbucket","source":"gitrepo","eventType":"branchDeleted","branchName":"kt4","repositoryName":"krishnakadiyam/jfrog-pipelines-second"}
    {"timestamp":"2022-06-13T05:29:55.062Z","domain":"webhook","status":"success","durationMillis":234,"webhookId":"2d4d698b-b083-42fd-a28e-670d9cec4c1a","integration":"glRepo","source":"gitrepo","eventType":"tag","repositoryName":"jfrog-pipelines-second","tagName":"refs/tags/kt4"}
  • Pipelines Integrations Events (Pipelines 1.29 and above):For every integrations activity, you will find the following metrics in api-activity-event.log.

    {"timestamp":"2022-11-10T10:36:50.004Z","domain":"projectIntegrations","eventType":"create","status":"success","integrationName":"iwh","integrationId":1,"integrationType":"incomingWebhook","createdBy":"admin","updatedBy":"admin","durationMillis":188}
    {"timestamp":"2022-11-10T10:37:43.423Z","domain":"projectIntegrations","eventType":"update","status":"success","integrationName":"iwh","integrationId":1,"integrationType":"incomingWebhook","createdBy":"admin","updatedBy":"admin","durationMillis":38}
    {"timestamp":"2022-11-10T10:37:55.901Z","domain":"projectIntegrations","eventType":"delete","status":"success","integrationName":"iwh","integrationId":"1","integrationType":"incomingWebhook","createdBy":"admin","updatedBy":"admin","durationMillis":85}