ARTIFACTORY: How to apply Artifactory tuning parameters when using Helm based Installations

Vignesh Surendrababu
2023-01-22 11:07

In certain cases, it is required to perform Vertical scaling on Artifactory related to database connections, maxThreads, etc., when the load on Artifactory is high.

When using a VM based installation, it is possible to directly update the system.yaml file with the required values as explained in this KB article. However, when using a helm based installation, it is not possible to edit the specific system.yaml present within the pod.

In this case, it is recommended to use the system.yaml secrets and use the custom values.yaml file used for deploying the Artifactory.

Example: https://github.com/jfrog/charts/blob/master/stable/artifactory/values.yaml#L71systemYamlOverride:
  existingSecret: system-yaml
  dataKey: system.yaml
 

What are secrets? 

A Secret is an object that contains sensitive data such as a password, a token, or a key. Such information might otherwise be put in a Pod specification or in a container image. By using a Secret, it is not required to include confidential data in your application code and the purpose of Secrets is to reduce the risk of exposing sensitive data while deploying applications on Kubernetes.

How to create secrets?

Step 1:

In order to create the secrets, firstly it is required to check the existing system.yaml used within the Artifactory pods. 

Hence, exec to the Artifactory pod and navigate to /opt/jfrog/artifactory/var/etc/ directory and copy the system.yaml file outside the pod and save it.

Step 2:

Now, we can use the existing system.yaml file available to modify the values with the required parameters such as database maxConnections, maxThreads, extraJavaOpts to update custom runtime property.

Existing system.yamlrouter:
  serviceRegistry:
    insecure: false
shared:
  logging:
    consoleLog:
      enabled: false
  extraJavaOpts: >
    -Dartifactory.access.client.max.connections=50
  database:
    type: postgresql
    url: "jdbc:postgresql://jfrt-postgresql:5432/artifactory"
    driver: org.postgresql.Driver
    username: "artifactory"
artifactory:
  database:
    maxOpenConnections: 80
  tomcat:
    maintenanceConnector:
      port: 8091
    connector:
      maxThreads: 200
      sendReasonPhrase: false
      extraConfig: acceptCount="100"
frontend:
  session:
    timeMinutes: "30"
access:
  database:
    maxOpenConnections: 80
  tomcat:
    connector:
      maxThreads: 50
      sendReasonPhrase: false
      extraConfig: acceptCount="100"
metadata:
  database:
    maxOpenConnections: 80
jfconnect:
  enabled: true
 

Step 3:

Update the system.yaml with increased database connections, tomcat threads etc
 router:
  serviceRegistry:
    insecure: false
shared:
  logging:
    consoleLog:
      enabled: false
  extraJavaOpts: >
    -Dartifactory.access.client.max.connections=250
  database:
    type: postgresql
    url: "jdbc:postgresql://jfrt-postgresql:5432/artifactory"
    driver: org.postgresql.Driver
    username: "artifactory"
artifactory:
  database:
    maxOpenConnections: 300
  tomcat:
    maintenanceConnector:
      port: 8091
    connector:
      maxThreads: 1000
      sendReasonPhrase: false
      extraConfig: acceptCount="100"
frontend:
  session:
    timeMinutes: "30"
access:
  database:
    maxOpenConnections: 300
  tomcat:
    connector:
      maxThreads: 250
      sendReasonPhrase: false
      extraConfig: acceptCount="100"
metadata:
  database:
    maxOpenConnections: 300
jfconnect:
  enabled: true
 

Step 4:

Create a secret from the updated system.yaml filekubectl create secret generic system-yaml --from-file ./system.yamlThen, the secret can be viewed from the “kubectl get secrets” command

Step 5:

Use the secret name, datakey under the systemYamlOverride of values.yaml filesystemYamlOverride:
  existingSecret: system-yaml
  dataKey: system.yaml
 

Step 6:

Perform a helm upgrade in order to use the updated secrets of system.yaml

How to update the existing secret?

If the system.yaml secrets are already in use and in case, if there is a need to modify the system.yaml again, It is also possible to update the secrets directly using the “kubectl edit secret <secret name>”

Example:jfrt-artifactory-0                       0/1     Running   0          81s
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  system.yaml: cm91dGVyOgogIHNlcnZpY2VSZWdpc3RyeToKICAgIGluc2VjdXJlOiBmYWxzZQpzaGFyZWQ6CiAgbG9nZ2luZzoKICAgIGNvbnNvbGVMb2c6CiAgICAgIGVuYWJsZWQ6IGZhbHNlCiAgZXh0cmFKYXZhT3B0czogPgogICAgLURhcnRpZmFjdG9yeS5hY2Nlc3MuY2xpZW50Lm1heC5jb25uZWN0aW9ucz01MAogIGRhdGFiYXNlOgogICAgdHlwZTogcG9zdGdyZXNxbAogICAgdXJsOiAiamRiYzpwb3N0Z3Jlc3FsOi8vamZydC1wb3N0Z3Jlc3FsOjU0MzIvYXJ0aWZhY3RvcnkiCiAgICBkcml2ZXI6IG9yZy5wb3N0Z3Jlc3FsLkRyaXZlcgogICAgdXNlcm5hbWU6ICJhcnRpZmFjdG9yeSIKYXJ0aWZhY3Rvcnk6CiAgZGF0YWJhc2U6CiAgICBtYXhPcGVuQ29ubmVjdGlvbnM6IDgwCiAgdG9tY2F0OgogICAgbWFpbnRlbmFuY2VDb25uZWN0b3I6CiAgICAgIHBvcnQ6IDgwOTEKICAgIGNvbm5lY3RvcjoKICAgICAgbWF4VGhyZWFkczogMjAwCiAgICAgIHNlbmRSZWFzb25QaHJhc2U6IGZhbHNlCiAgICAgIGV4dHJhQ29uZmlnOiBhY2NlcHRDb3VudD0iMTAwIgpmcm9udGVuZDoKICBzZXNzaW9uOgogICAgdGltZU1pbnV0ZXM6ICIzMCIKYWNjZXNzOgogIGRhdGFiYXNlOgogICAgbWF4T3BlbkNvbm5lY3Rpb25zOiA4MAogIHRvbWNhdDoKICAgIGNvbm5lY3RvcjoKICAgICAgbWF4VGhyZWFkczogNTAKICAgICAgc2VuZFJlYXNvblBocmFzZTogZmFsc2UKICAgICAgZXh0cmFDb25maWc6IGFjY2VwdENvdW50PSIxMDAiCm1ldGFkYXRhOgogIGRhdGFiYXNlOgogICAgbWF4T3BlbkNvbm5lY3Rpb25zOiA4MApqZmNvbm5lY3Q6CiAgZW5hYmxlZDogdHJ1ZQo=
kind: Secret
metadata:
  annotations:
    meta.helm.sh/release-name: jfrt
    meta.helm.sh/release-namespace: default
  creationTimestamp: "2022-12-14T08:12:19Z"
  labels:
    app: artifactory
    app.kubernetes.io/managed-by: Helm
    chart: artifactory-107.47.12
    heritage: Helm
    release: jfrt
  name: jfrt-artifactory-systemyaml
  namespace: default
  resourceVersion: "31547656"
  uid: cbab4b27-0500-4e15-9f6a-171ca11c2238
type: Opaque

Here, in the above example, the dataKey.system.yaml content is in encoded format and we need to decode the content. 

Content to be decoded from above example: 

cm91dGVyOgogIHNlcnZpY2VSZWdpc3RyeToKICAgIGluc2VjdXJlOiBmYWxzZQpzaGFyZWQ6CiAgbG9nZ2luZzoKICAgIGNvbnNvbGVMb2c6CiAgICAgIGVuYWJsZWQ6IGZhbHNlCiAgZXh0cmFKYXZhT3B0czogPgogICAgLURhcnRpZmFjdG9yeS5hY2Nlc3MuY2xpZW50Lm1heC5jb25uZWN0aW9ucz01MAogIGRhdGFiYXNlOgogICAgdHlwZTogcG9zdGdyZXNxbAogICAgdXJsOiAiamRiYzpwb3N0Z3Jlc3FsOi8vamZydC1wb3N0Z3Jlc3FsOjU0MzIvYXJ0aWZhY3RvcnkiCiAgICBkcml2ZXI6IG9yZy5wb3N0Z3Jlc3FsLkRyaXZlcgogICAgdXNlcm5hbWU6ICJhcnRpZmFjdG9yeSIKYXJ0aWZhY3Rvcnk6CiAgZGF0YWJhc2U6CiAgICBtYXhPcGVuQ29ubmVjdGlvbnM6IDgwCiAgdG9tY2F0OgogICAgbWFpbnRlbmFuY2VDb25uZWN0b3I6CiAgICAgIHBvcnQ6IDgwOTEKICAgIGNvbm5lY3RvcjoKICAgICAgbWF4VGhyZWFkczogMjAwCiAgICAgIHNlbmRSZWFzb25QaHJhc2U6IGZhbHNlCiAgICAgIGV4dHJhQ29uZmlnOiBhY2NlcHRDb3VudD0iMTAwIgpmcm9udGVuZDoKICBzZXNzaW9uOgogICAgdGltZU1pbnV0ZXM6ICIzMCIKYWNjZXNzOgogIGRhdGFiYXNlOgogICAgbWF4T3BlbkNvbm5lY3Rpb25zOiA4MAogIHRvbWNhdDoKICAgIGNvbm5lY3RvcjoKICAgICAgbWF4VGhyZWFkczogNTAKICAgICAgc2VuZFJlYXNvblBocmFzZTogZmFsc2UKICAgICAgZXh0cmFDb25maWc6IGFjY2VwdENvdW50PSIxMDAiCm1ldGFkYXRhOgogIGRhdGFiYXNlOgogICAgbWF4T3BlbkNvbm5lY3Rpb25zOiA4MApqZmNvbm5lY3Q6CiAgZW5hYmxlZDogdHJ1ZQo= 

Decoded content:

router:
  serviceRegistry:
    insecure: false
shared:
  logging:
    consoleLog:
      enabled: false
  extraJavaOpts: >
    -Dartifactory.access.client.max.connections=50
  database:
    type: postgresql
    url: "jdbc:postgresql://jfrt-postgresql:5432/artifactory"
    driver: org.postgresql.Driver
    username: "artifactory"
artifactory:
  database:
    maxOpenConnections: 80
  tomcat:
    maintenanceConnector:
      port: 8091
    connector:
      maxThreads: 200
      sendReasonPhrase: false
      extraConfig: acceptCount="100"
frontend:
  session:
    timeMinutes: "30"
access:
  database:
    maxOpenConnections: 80
  tomcat:
    connector:
      maxThreads: 50
      sendReasonPhrase: false
      extraConfig: acceptCount="100"
metadata:
  database:
    maxOpenConnections: 80
jfconnect:
  enabled: true

Note: By default the secrets will be encoded and decoded using the base64 format.

Hence, we can prefer using any external encoding/decoding tools to convert the string available in the dataKey.system.yaml.

Once it is decoded, update the system.yaml content with the necessary changes and then encode the yaml file content again as suggested above.

Here, we have updated the system.yaml content with following values shown below:router:
  serviceRegistry:
    insecure: false
shared:
  logging:
    consoleLog:
      enabled: false
  extraJavaOpts: >
    -Dartifactory.access.client.max.connections=250 -Dartifactory.async.corePoolSize=16
  database:
    type: postgresql
    url: "jdbc:postgresql://jfrt-postgresql:5432/artifactory"
    driver: org.postgresql.Driver
    username: "artifactory"
artifactory:
  database:
    maxOpenConnections: 300
  tomcat:
    maintenanceConnector:
      port: 8091
    connector:
      maxThreads: 1200
      sendReasonPhrase: false
      extraConfig: acceptCount="100"
frontend:
  session:
    timeMinutes: "30"
access:
  database:
    maxOpenConnections: 300
  tomcat:
    connector:
      maxThreads: 250
      sendReasonPhrase: false
      extraConfig: acceptCount="100"
metadata:
  database:
    maxOpenConnections: 300
jfconnect:
  enabled: true

We will encode the above updated system YAML file content again:cm91dGVyOgogIHNlcnZpY2VSZWdpc3RyeToKICAgIGluc2VjdXJlOiBmYWxzZQpzaGFyZWQ6CiAg
bG9nZ2luZzoKICAgIGNvbnNvbGVMb2c6CiAgICAgIGVuYWJsZWQ6IGZhbHNlCiAgZXh0cmFKYXZh
T3B0czogPgogICAgLURhcnRpZmFjdG9yeS5hY2Nlc3MuY2xpZW50Lm1heC5jb25uZWN0aW9ucz0y
NTAgLURhcnRpZmFjdG9yeS5hc3luYy5jb3JlUG9vbFNpemU9MTYKICBkYXRhYmFzZToKICAgIHR5
cGU6IHBvc3RncmVzcWwKICAgIHVybDogImpkYmM6cG9zdGdyZXNxbDovL2pmcnQtcG9zdGdyZXNx
bDo1NDMyL2FydGlmYWN0b3J5IgogICAgZHJpdmVyOiBvcmcucG9zdGdyZXNxbC5Ecml2ZXIKICAg
IHVzZXJuYW1lOiAiYXJ0aWZhY3RvcnkiCmFydGlmYWN0b3J5OgogIGRhdGFiYXNlOgogICAgbWF4
T3BlbkNvbm5lY3Rpb25zOiAzMDAKICB0b21jYXQ6CiAgICBtYWludGVuYW5jZUNvbm5lY3RvcjoK
ICAgICAgcG9ydDogODA5MQogICAgY29ubmVjdG9yOgogICAgICBtYXhUaHJlYWRzOiAxMjAwCiAg
ICAgIHNlbmRSZWFzb25QaHJhc2U6IGZhbHNlCiAgICAgIGV4dHJhQ29uZmlnOiBhY2NlcHRDb3Vu
dD0iMTAwIgpmcm9udGVuZDoKICBzZXNzaW9uOgogICAgdGltZU1pbnV0ZXM6ICIzMCIKYWNjZXNz
OgogIGRhdGFiYXNlOgogICAgbWF4T3BlbkNvbm5lY3Rpb25zOiAzMDAKICB0b21jYXQ6CiAgICBj
b25uZWN0b3I6CiAgICAgIG1heFRocmVhZHM6IDI1MAogICAgICBzZW5kUmVhc29uUGhyYXNlOiBm
YWxzZQogICAgICBleHRyYUNvbmZpZzogYWNjZXB0Q291bnQ9IjEwMCIKbWV0YWRhdGE6CiAgZGF0
YWJhc2U6CiAgICBtYXhPcGVuQ29ubmVjdGlvbnM6IDMwMApqZmNvbm5lY3Q6CiAgZW5hYmxlZDog
dHJ1ZQo=

As a next step, edit the secret (kubectl edit secret <secret name>) and replace the new encoded key under the dataKey.system.yaml, then save it (:wq).

User-added image

Finally, restart the Artifactory pods for the changes to take effect.