ARTIFACTORY: Splunk Artifactory integration via helm charts in Kubernetes environment

ARTIFACTORY: Splunk Artifactory integration via helm charts in Kubernetes environment

AuthorFullName__c
Shivani Budhodi
articleNumber
000006101
ft:sourceType
Salesforce
FirstPublishedDate
2024-05-22T07:23:36Z
lastModifiedDate
2024-05-22
VersionNumber
2
1.  Add the Jfrog helm repository:
helm repo add jfrog https://charts.jfrog.io
helm repo update
2. Install Artifactory using the values.yaml file mentioned here, it will also spin up the fluentd side-car container.

Values.yaml file for Artifactory HA:
installerInfo: '{ "productId": "OnPremObservability-Splunk/1.0.0", "features": [ { "featureId": "ArtifactoryVersion/{{ default .Chart.AppVersion .Values.artifactory.image.version }}" }, { "featureId": "{{ if .Values.postgresql.enabled }}postgresql{{ else }}{{ .Values.database.type }}{{ end }}/0.0.0" }, { "featureId": "Platform/{{ default "kubernetes" .Values.installer.platform }}" },  { "featureId": "Channel/OnPremObservability-Splunk-Helm" } ] }'
artifactory:
  masterKey: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
  joinKey: EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
  customInitContainersBegin: |
    - name: "prepare-fluentd-conf-on-persistent-volume"
      image: "{{ .Values.initContainerImage }}"
      imagePullPolicy: "{{ .Values.artifactory.image.pullPolicy }}"
      command:
        - 'sh'
        - '-c'
        - >
          mkdir -p {{ .Values.artifactory.persistence.mountPath }}/etc/fluentd/;
          curl https://raw.githubusercontent.com/jfrog/log-analytics-splunk/master/fluent.conf.rt -o {{ .Values.artifactory.persistence.mountPath }}/etc/fluentd/fluentd.conf
      volumeMounts:
        - mountPath: "{{ .Values.artifactory.persistence.mountPath }}"
          name: volume
  customSidecarContainers: |
    - name: "artifactory-fluentd-sidecar"
      image: "releases-pts-observability-fluentd.jfrog.io/fluentd:2.0"
      imagePullPolicy: "IfNotPresent"
      volumeMounts:
        - mountPath: "{{ .Values.artifactory.persistence.mountPath }}"
          name: volume
      env:
        - name: JF_PRODUCT_DATA_INTERNAL
          value: {{ .Values.artifactory.persistence.mountPath }}
        - name: JPD_URL
          value: {{ .Values.jfrog.observability.jpd_url }}
        - name: JPD_ADMIN_USERNAME
          value: {{ .Values.jfrog.observability.username }}
        - name: JFROG_ADMIN_TOKEN
          value: {{ .Values.jfrog.observability.access_token }}
        - name: COMMON_JPD
          value: {{ .Values.jfrog.observability.common_jpd | quote }}
        - name: SPLUNK_COM_PROTOCOL
          value: {{ .Values.splunk.com_protocol }}
        - name: SPLUNK_HEC_HOST
          value: {{ .Values.splunk.host }}
        - name: SPLUNK_HEC_PORT
          value: {{ .Values.splunk.port | quote}}
        - name: SPLUNK_METRICS_HEC_TOKEN
          value: {{ .Values.splunk.metrics_token }}
        - name: SPLUNK_HEC_TOKEN
          value: {{ .Values.splunk.logs_token }}
        - name: SPLUNK_INSECURE_SSL
          value: {{ .Values.splunk.insecure_ssl | quote}}
        - name: FLUENTD_CONF
          value: ../../../../{{ .Values.artifactory.persistence.mountPath }}/etc/fluentd/fluentd.conf
splitServicesToContainers: false
splunk:
  host: SPLUNK_HEC_HOST
  port: SPLUNK_HEC_PORT
  logs_token: SPLUNK_HEC_TOKEN
  metrics_token: SPLUNK_METRICS_HEC_TOKEN
  com_protocol: SPLUNK_COM_PROTOCOL
  insecure_ssl: SPLUNK_INSECURE_SSL
jfrog:
  observability:
    jpd_url: JPD_URL
    username: JPD_ADMIN_USERNAME
    access_token: JFROG_ADMIN_TOKEN
    common_jpd: COMMON_JPD
postgresql:
  postgresqlPassword: Password@123
databaseUpgradeReady: true
unifiedUpgradeAllowed: true
nginx:
  enabled: true

3. Install Artifactory using the below command:
helm upgrade --install artifactory-ha  -- namespace artifactory-ha jfrog-charts/artifactory-ha  -f artifactory-values.yaml

4. Download the Splunk Application from the Splunk base or you may use the splunk.yaml file mentioned on our GitHub page.
Splunk.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: splunk
  name: splunk
  #namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: splunk
  template:
    metadata:
      labels:
        app: splunk
    spec:
      containers:
        - image: splunk/splunk:latest
          imagePullPolicy: "Always"
          name: splunk
          env:
            - name: SPLUNK_START_ARGS
              value: "--accept-license"
            - name: SPLUNK_PASSWORD
              value: "password1"
          ports:
            -  containerPort: 8000
            -  containerPort: 8088
            -  containerPort: 8089   
---
kind: Service
apiVersion: v1
metadata:
  name: splunk
  labels:
    app: splunk
spec:
  selector:
    app: splunk
  ports:
  - name: web
    protocol: TCP
    port: 8000
    targetPort: 8000
  - name: hec
    protocol: TCP
    port: 8088
    targetPort: 8088
  - name: api
    protocol: TCP
    port: 8089
    targetPort: 8089
  type: ClusterIP
---
kind: Service
apiVersion: v1
metadata:
  name: splunk-lb
  labels:
    app: splunk
spec:
  selector:
    app: splunk
  ports:
  - name: http 
    protocol: TCP
    port: 80
    targetPort: 8000
  - name: hec
    protocol: TCP
    port: 8088
    targetPort: 8088
  - name: api
    protocol: TCP
    port: 8089
    targetPort: 8089
  type: LoadBalancer

5. Execute the below command to install splunk from the specified splunk.yaml:
kubectl apply -f k8s/splunk.yaml
User-added image

6. The default Splunk user name and password will be admin and password1 respectively.

7. Login to the Splunk server. Now, we need to create the indexes jfrog_splunk and jfrog_splunk_metrics by following the below steps.
a. Create index jfrog_splunk
i. Navigate to the Splunk console as Administrator, Click on "Settings" in the drop-down, and select "Indexes". 
ii.  Now create a new index by clicking on "New Index", enter the Index name as jfrog_splunk,  and save it.
b. Create index jfrog_splunk_metrics
i. Again create a new index by clicking on "New Index", enter the Index name as Jfrog_splunk_metrics, but this time while creating a new index we have to select Index Data Type as Metrics.
ii. Click on save.

8. Navigate to Settings-> Data inputs-> HTTP Event Collector. Before creating the HEC token , kindly edit the global setting by selecting "Default Source Type” as generic_single_line and save it.

User-added image

User-added image

User-added image
9. Now create an HEC token to receive Logs, click on the new token button. Give a name to the token and click next, then select the jfrog_splunk index and click on review and submit.

User-added image

10. Create another HEC token to receive Metrics. Once again, click on the new token, enter a name for the token, and select jfrog_splunk_metrics as index. Then click on the review button and submit.

User-added image

User-added image

Note: The source type should be generic_single_line for both tokens.

11. Create a .env_jfrog file with the below content:
export SPLUNK_COM_PROTOCOL=http/https
export SPLUNK_HEC_HOST=<splunk_ip>
export SPLUNK_HEC_PORT=8088
export SPLUNK_HEC_TOKEN=<token generated for splunk_jfrog index>
export SPLUNK_METRICS_HEC_TOKEN=<token generated for splunk_jfrog_metrics index>
export SPLUNK_INSECURE_SSL=false/true
export JPD_URL=http://<artifactory_ip>
export JPD_ADMIN_USERNAME= <username>
export JFROG_ADMIN_TOKEN= <artifactory_token>
export COMMON_JPD=false

12. Execute the below command:
      source .env_jfrog

13. Now perform a helm upgrade using the below command:
helm upgrade --install artifactory-ha  jfrog/artifactory-ha \
       --set artifactory.masterKey=FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF \
       --set artifactory.joinKey=EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE \
       --set splunk.host=$SPLUNK_HEC_HOST \
       --set splunk.port=$SPLUNK_HEC_PORT \
       --set splunk.logs_token=$SPLUNK_HEC_TOKEN \
       --set splunk.metrics_token=$SPLUNK_METRICS_HEC_TOKEN \
       --set splunk.com_protocol=$SPLUNK_COM_PROTOCOL \
       --set splunk.insecure_ssl=$SPLUNK_INSECURE_SSL \
       --set jfrog.observability.jpd_url=$JPD_URL \
       --set jfrog.observability.username=$JPD_ADMIN_USERNAME \
       --set jfrog.observability.access_token=$JFROG_ADMIN_TOKEN \
       --set jfrog.observability.common_jpd=$COMMON_JPD \
       -f helm/artifactory-ha-values.yaml

14. After performing the helm upgrade, the Artifactory metrics will be visible in the Splunk UI.

image (80).png