The following are the three metric log files created for Pipelines:
Open Metrics Format:
Pipeline API Metrics:
$JF_PRODUCT_HOME/var/log/api-metrics.log
Non-Open Metrics Format:
Pipeline Reqsealer Event Metrics:
$JF_PRODUCT_HOME/var/log/reqsealer-activity-event.log
Pipeline Sync Event Metrics:
$JF_PRODUCT_HOME/var/log/pipelinesync-activity-event.log
Pipelines Open Metrics Format
The Get Pipelines Metrics Data REST API returns the following metrics in Open Metrics format.
Metric | Description |
---|---|
sys_cpu_user_seconds | User CPU usage time for thethe pipelineprocess in seconds |
sys_cpu_system_seconds | System CPU usage time forthe pipelineprocess in seconds |
sys_cpu_total_seconds | Total CPU usage time for the pipeline process in seconds |
nodejs_heap_read_only_space_total | Total size allocated for V8 heap segment “read_only_space” |
nodejs_heap_read_only_space_used_total | Used size for V8 heap segment “read_only_space” |
nodejs_heap_new_space_total | Total size allocated for V8 heap segment “new_space” |
nodejs_heap_new_space_used_total | Used size for V8 heap segment “new_space” |
nodejs_heap_old_space_total | Total size allocated for V8 heap segment “old_space” |
nodejs_heap_old_space_used_total | Used size for V8 heap segment “old_space” |
nodejs_heap_code_space_total | Total size allocated for V8 heap segment “code_space” |
nodejs_heap_code_space_used_total | Used size for V8 heap segment “code_space” |
nodejs_heap_map_space_total | Total size allocated for V8 heap segment “max_space” |
nodejs_heap_map_space_used_total | Used size for V8 heap segment “max_space” |
nodejs_heap_large_object_space_total | Total size allocated for V8 heap segment “large_object_space” |
nodejs_heap_large_object_space_used_total | Used size for V8 heap segment “large_object_space” |
nodejs_heap_code_large_object_space_total | Total size allocated for V8 heap segment “code_large_object_space” |
nodejs_heap_code_large_object_space_used_total | Used size for V8 heap segment “code_large_object_space” |
nodejs_heap_new_large_object_space_total | Total size allocated for V8 heap segment “new_large_object_space” |
nodejs_heap_new_large_object_space_used_total | Used size for V8 heap segment “new_large_object_space” |
sys_memory_free_bytes | Host free virtual memory |
sys_memory_total_bytes | Host total virtual memory |
jfpip_pipelines_per_project_count NoteIn Pipelines 1.24 and prior, this is called | Number of Pipelines Per Project |
jfpip_pipelines_count NoteIn Pipelines 1.24 and prior, this is called | Number of Total Pipelines |
jfpip_queue_messages_total_count | Messages Count for the Queue |
jfpip_nodepool_provisionstatus_success_count | Number of node with SUCCESS provisioned status |
jfpip_nodepool_provisionstatus_cached_count | Number of node with CACHED provisioned status |
jfpip_nodepool_provisionstatus_processing_count | Number of node with PROCESSING provisioned status |
jfpip_nodepool_provisionstatus_failure_count | Number of node with FAILURE provisioned status |
jfpip_nodepool_provisionstatus_waiting_count | Number of node with WAITING provisioned status |
jfpip_concurrent_active_builds_count | Active Concurrent Build Count |
jfpip_concurrent_allowed_builds_count | Allowed Concurrent Build Count |
jfpip_concurrent_available_builds_count | Available Concurrent Build Count |
Note
All node.js
heap size statistics are captured using the v8.getHeapSpaceStatistics()API.
Pipeline Logs
The api-metrics.log
will contain system metrics such as:
Total disk space used
Total disk space free
Time CPU is used by the process
Node JS Heap related information
Non-Open Metrics Format
In addition to the metrics mentioned about, Pipelines supports the following custom activity-based Event Metrics:
Pipeline Run & Step Events: For every pipeline run, two types of metrics can be found in
reqsealer-activity-event.log
. One entry for each step status and one entry for overall pipeline status.{"timestamp":"2022-04-05T08:30:10.088Z","startedAt":"2022-04-05T08:30:03.986Z","queuedAt":"2022-04-05T08:30:03.010Z","domain":"step","pipelineName":"my_pipeline_2","triggeredBy":"admin","branchName":"master","stepName":"p2_s1","runNumber":2,"status":"success","durationMillis":6102,"outputArtifactsCount":0,"outputResourcesCount":0} {"timestamp":"2022-04-05T08:30:10.088Z","startedAt":"2022-04-05T08:30:03.986Z","domain":"run","pipelineName":"my_pipeline_2","triggeredBy":"admin","branchName":"master","runNumber":2,"status":"success","durationMillis":6102}
Pipeline Sync Events:For every pipeline sync activity, the following metrics can be found in
pipelinesync-activity-event.log
.{"timestamp":"2022-04-06T10:00:45.673Z","domain":"pipelineSync","pipelineSourceName":"Sample","repositoryName":"a-0908/myFirstRepo","branch":"master","status":"success","durationMillis":10498}
Webhook Events (Pipelines 1.25 and above):For every pipeline sync activity, the following metrics can be found in
pipelinesync-activity-event.log
.{"timestamp":"2022-06-10T16:29:29.894Z","domain":"webhook","status":"success","durationMillis":533,"webhookId":"11819184-2d88-4180-92da-aa13092d0ca4","integration":"my_bitbucket","source":"gitrepo","eventType":"branchCreated","branchName":"kt4","repositoryName":"krishnakadiyam/jfrog-pipelines-second"} {"timestamp":"2022-06-10T16:29:40.845Z","domain":"webhook","status":"success","durationMillis":323,"webhookId":"6d098e3a-7b4b-427c-ba53-b1174baeeabd","integration":"my_bitbucket","source":"gitrepo","eventType":"branchDeleted","branchName":"kt4","repositoryName":"krishnakadiyam/jfrog-pipelines-second"} {"timestamp":"2022-06-13T05:29:55.062Z","domain":"webhook","status":"success","durationMillis":234,"webhookId":"2d4d698b-b083-42fd-a28e-670d9cec4c1a","integration":"glRepo","source":"gitrepo","eventType":"tag","repositoryName":"jfrog-pipelines-second","tagName":"refs/tags/kt4"}
Pipelines Integrations Events (Pipelines 1.29 and above):For every integrations activity, you will find the following metrics in
api-activity-event.log
.{"timestamp":"2022-11-10T10:36:50.004Z","domain":"projectIntegrations","eventType":"create","status":"success","integrationName":"iwh","integrationId":1,"integrationType":"incomingWebhook","createdBy":"admin","updatedBy":"admin","durationMillis":188} {"timestamp":"2022-11-10T10:37:43.423Z","domain":"projectIntegrations","eventType":"update","status":"success","integrationName":"iwh","integrationId":1,"integrationType":"incomingWebhook","createdBy":"admin","updatedBy":"admin","durationMillis":38} {"timestamp":"2022-11-10T10:37:55.901Z","domain":"projectIntegrations","eventType":"delete","status":"success","integrationName":"iwh","integrationId":"1","integrationType":"incomingWebhook","createdBy":"admin","updatedBy":"admin","durationMillis":85}