Prometheus with Grafana

JFrog Platform Administration Documentation
Content Type
Administration / Platform
ft:sourceType
Paligo

Subscription Information

This feature is supported on the Self-Hosted platform, with a Pro, Pro X, Enterprise X, or Enterprise+ license.

JFrog supports integration with Prometheus for log monitoring. The JFrog Log Analytics and Metrics solution using Prometheus consists of three segments,

  1. Prometheus - the component where metrics data gets ingested

  2. Loki - the component where log data gets ingested

  3. Grafana - the component where data visualization is achieved via prebuilt dashboards

Note

This version replaces all previous implementations. This version is not an in-place upgrade to the existing solution from JFrog but a full reinstall. Any dashboard customizations done on previous versions will need to be redone after this installation

Note

This guide assumes the implementer is performing a new setup. Changes to handle install in an existing setup are highlighted where applicable. If Prometheus is already installed and configured, we recommend to have the existing Prometheus release name handy. If Loki is already installed and configured, we recommend to have its service URL handy.

If Prometheus and Loki are already available, you can skip the installation section and proceed to the configuration section.

Prerequisites for Prometheus with Grafana

  • Working and configured Kubernetes Cluster - Amazon EKS / Google GKE / Azure AKS / Docker Desktop / Minikube

  • 'kubectl' utility on the workstation which is capable of connecting to the Kubernetes cluster

  • HELM v3 Installed

  • Versions supported and tested:

    • Jfrog Platform: 10.9.2

    • Artifactory: 7.46.10

    • Xray: 3.59.4

    • Prometheus: 2.39.1

    • Grafana: 9.2.3

    • Loki: 2.6.1

Install Prometheus, Loki and Grafana on Kubernetes

The Prometheus Community kube-prometheus-stack helm chart allows the creation of Prometheus instances and includes Grafana. The Grafana Community grafana helm chart allows the creation of Loki instances and includes Grafana that can link to Prometheus.

Once the prerequisites are met, to install Prometheus Kubernetes stack:

Create the Namespace required for Prometheus Stack deployment
kubectl create namespace jfrog-plg

We will use jfrog-plg as the namespace throughout this document.

Install the Prometheus Chart

  1. Add the required Helm Repositories:

    helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

  2. Install the chart:

    helm install "prometheus" prometheus-community/kube-prometheus-stack -n jfrog-plg

    Where: "prometheus" is the value that needs to be used against the value for "release_name" in the configuration section.

For Docker Desktop, run this additional command to correct the mount path propagation for Prometheus node-exporter component. An error event will appear as follows "Error: failed to start container "node-exporter": Error response from daemon: path / is mounted on / but it is not a shared or slave mount".

kubectl patch ds prometheus-prometheus-node-exporter --type "json" -p '[{"op": "remove", "path" : "/spec/template/spec/containers/0/volumeMounts/2/mountPropagation"}]' -n jfrog-plg
Install the Loki Chart

  1. Add the required Helm Repositories:

    helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
 
helm repo add loki https://grafana.github.io/loki/charts
helm repo update

  2. Install the chart:

    helm upgrade --install "loki" grafana/loki-stack -n jfrog-plg

    Where:

    • "loki" will be the service name. The url to access Loki as a datasource can be visualised as http://<service_name>.<namespace>:<port>. Example: http://loki.jfrog-plg:3100 will be the "loki_url" value.

    • --version v2.6.1 is the most recent Loki version at the time of writing the document. If there is a need to deploy this exact version, then add "--version v2.6.1" to the command line.

Promotheus with Grafana Configuration

The configuration that is needed before the JFrog products installation can take place are listed below:

From any of the value files for applying charts, that is in helm/jfrog-platform-values.yaml, helm/artifactory-values.yaml, helm/artifactory-ha-values.yaml or helm/xray-values.yaml download and provide the following values for global.prometheus.loki_url and global.prometheus.release_name as per the installation if Prometheus and Loki are already installed:

global:
   jfrog:
      observability:
         branch: master
   prometheus:
      loki_url: http:\/\/loki.jfrog-plg:3100
      release_name: prometheus
Prometheus with Grafana - JFrog Platform + Metrics via Helm

Ensure JFrog repo is added to helm.

helm repo add jfrog https://charts.jfrog.io
helm repo update

To configure and install JFrog Platform with Prometheus metrics being exposed use our file helm/jfrog-platform-values.yaml to expose a metrics and new service monitor to Prometheus.

JFrog Platform:

helm upgrade --install jfrog-platform jfrog/jfrog-platform \
       -f helm/jfrog-platform-values.yaml \
       -n jfrog-plg

Note

If you are installing in the same cluster with the deprecated solution, Use the same namespace as the previous one instead of jfrog-plg above.

Prometheus with Grafana - Artifactory / Artifactory HA + Metrics via Helm

For configuring and installing Artifactory Pro/Pro-x use the artifactory-values.yaml file.

For configuring and installing Enterprise/Ent+ use the artifactory-ha-values.yaml file.

You can apply them to your helm install examples below:

Artifactory:

helm upgrade --install artifactory jfrog/artifactory \
       --set artifactory.masterKey=FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF \
       --set artifactory.joinKey=EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE \
       -f helm/artifactory-values.yaml \
       -n jfrog-plg

If you are installing in the same cluster with the deprecated solution, Use the same namespace as the previous one instead of jfrog-plg above.

Artifactory-HA:

helm upgrade --install artifactory-ha jfrog/artifactory-ha \
       --set artifactory.masterKey=FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF \
       --set artifactory.joinKey=EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE \
       -f helm/artifactory-ha-values.yaml \
       -n jfrog-plg

If you are installing in the same cluster with the deprecated solution, Use the same namespace as the previous one instead of jfrog-plg above.

Note the above examples are only references you will need additional parameters to configure TLS, binary blob storage, or other common Artifactory features.

This will complete the necessary configuration for Artifactory and expose a new service monitor servicemonitor-artifactory to expose metrics to Prometheus.

Promotheus with Grafana - Xray + Metrics via Helm

To configure and install Xray with Prometheus metrics being exposed use our file helm/xray-values.yaml to expose a metrics and new service monitor to Prometheus.

Xray:

helm upgrade --install xray jfrog/xray --set xray.jfrogUrl=http://my-artifactory-nginx-url \
       --set xray.masterKey=FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF \
       --set xray.joinKey=EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE \
       -f helm/xray-values.yaml \
       -n jfrog-plg

If you are installing in the same cluster with the deprecated solution, Use the same namespace as the previous one instead of jfrog-plg above.

Grafana Dashboard

Example dashboards are included in the grafana directory. These dashboards needs to be imported to the grafana. These include:

Assess the Promotheus with Grafana Setup

Use kubectl port forwards as mentioned in two terminal windows.

kubectl port-forward service/prometheus-operated 9090:9090 -n jfrog-plg
kubectl port-forward service/prometheus-grafana 3000:80 -n jfrog-plg

  1. Go to the web UI of the Prometheus instance "http://localhost:9090" and verify "Status -> Service Discovery", the list shows the new ServiceMonitor for Artifactory or Xray or Both.