ARTIFACTORY: How to apply Artifactory tuning parameters when using Helm based Installations
2023-01-22 11:07
In certain cases, it is required to perform Vertical scaling on Artifactory related to database connections, maxThreads, etc., when the load on Artifactory is high.
When using a VM based installation, it is possible to directly update the system.yaml file with the required values as explained in this KB article. However, when using a helm based installation, it is not possible to edit the specific system.yaml present within the pod.
In this case, it is recommended to use the system.yaml secrets and use the custom values.yaml file used for deploying the Artifactory.
Example: https://github.com/jfrog/charts/blob/master/stable/artifactory/values.yaml#L71systemYamlOverride:
existingSecret: system-yaml
dataKey: system.yaml
What are secrets?
A Secret is an object that contains sensitive data such as a password, a token, or a key. Such information might otherwise be put in a Pod specification or in a container image. By using a Secret, it is not required to include confidential data in your application code and the purpose of Secrets is to reduce the risk of exposing sensitive data while deploying applications on Kubernetes.
How to create secrets?
Step 1:
In order to create the secrets, firstly it is required to check the existing system.yaml used within the Artifactory pods.
Hence, exec to the Artifactory pod and navigate to /opt/jfrog/artifactory/var/etc/ directory and copy the system.yaml file outside the pod and save it.
Step 2:
Now, we can use the existing system.yaml file available to modify the values with the required parameters such as database maxConnections, maxThreads, extraJavaOpts to update custom runtime property.
Existing system.yamlrouter:
serviceRegistry:
insecure: false
shared:
logging:
consoleLog:
enabled: false
extraJavaOpts: >
-Dartifactory.access.client.max.connections=50
database:
type: postgresql
url: "jdbc:postgresql://jfrt-postgresql:5432/artifactory"
driver: org.postgresql.Driver
username: "artifactory"
artifactory:
database:
maxOpenConnections: 80
tomcat:
maintenanceConnector:
port: 8091
connector:
maxThreads: 200
sendReasonPhrase: false
extraConfig: acceptCount="100"
frontend:
session:
timeMinutes: "30"
access:
database:
maxOpenConnections: 80
tomcat:
connector:
maxThreads: 50
sendReasonPhrase: false
extraConfig: acceptCount="100"
metadata:
database:
maxOpenConnections: 80
jfconnect:
enabled: true
Step 3:
Update the system.yaml with increased database connections, tomcat threads etc
router:
serviceRegistry:
insecure: false
shared:
logging:
consoleLog:
enabled: false
extraJavaOpts: >
-Dartifactory.access.client.max.connections=250
database:
type: postgresql
url: "jdbc:postgresql://jfrt-postgresql:5432/artifactory"
driver: org.postgresql.Driver
username: "artifactory"
artifactory:
database:
maxOpenConnections: 300
tomcat:
maintenanceConnector:
port: 8091
connector:
maxThreads: 1000
sendReasonPhrase: false
extraConfig: acceptCount="100"
frontend:
session:
timeMinutes: "30"
access:
database:
maxOpenConnections: 300
tomcat:
connector:
maxThreads: 250
sendReasonPhrase: false
extraConfig: acceptCount="100"
metadata:
database:
maxOpenConnections: 300
jfconnect:
enabled: true
Step 4:
Create a secret from the updated system.yaml filekubectl create secret generic system-yaml --from-file ./system.yamlThen, the secret can be viewed from the “kubectl get secrets” command
Step 5:
Use the secret name, datakey under the systemYamlOverride of values.yaml filesystemYamlOverride:
existingSecret: system-yaml
dataKey: system.yaml
Step 6:
Perform a helm upgrade in order to use the updated secrets of system.yaml
How to update the existing secret?
If the system.yaml secrets are already in use and in case, if there is a need to modify the system.yaml again, It is also possible to update the secrets directly using the “kubectl edit secret <secret name>”
Example:jfrt-artifactory-0 0/1 Running 0 81s
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
system.yaml: cm91dGVyOgogIHNlcnZpY2VSZWdpc3RyeToKICAgIGluc2VjdXJlOiBmYWxzZQpzaGFyZWQ6CiAgbG9nZ2luZzoKICAgIGNvbnNvbGVMb2c6CiAgICAgIGVuYWJsZWQ6IGZhbHNlCiAgZXh0cmFKYXZhT3B0czogPgogICAgLURhcnRpZmFjdG9yeS5hY2Nlc3MuY2xpZW50Lm1heC5jb25uZWN0aW9ucz01MAogIGRhdGFiYXNlOgogICAgdHlwZTogcG9zdGdyZXNxbAogICAgdXJsOiAiamRiYzpwb3N0Z3Jlc3FsOi8vamZydC1wb3N0Z3Jlc3FsOjU0MzIvYXJ0aWZhY3RvcnkiCiAgICBkcml2ZXI6IG9yZy5wb3N0Z3Jlc3FsLkRyaXZlcgogICAgdXNlcm5hbWU6ICJhcnRpZmFjdG9yeSIKYXJ0aWZhY3Rvcnk6CiAgZGF0YWJhc2U6CiAgICBtYXhPcGVuQ29ubmVjdGlvbnM6IDgwCiAgdG9tY2F0OgogICAgbWFpbnRlbmFuY2VDb25uZWN0b3I6CiAgICAgIHBvcnQ6IDgwOTEKICAgIGNvbm5lY3RvcjoKICAgICAgbWF4VGhyZWFkczogMjAwCiAgICAgIHNlbmRSZWFzb25QaHJhc2U6IGZhbHNlCiAgICAgIGV4dHJhQ29uZmlnOiBhY2NlcHRDb3VudD0iMTAwIgpmcm9udGVuZDoKICBzZXNzaW9uOgogICAgdGltZU1pbnV0ZXM6ICIzMCIKYWNjZXNzOgogIGRhdGFiYXNlOgogICAgbWF4T3BlbkNvbm5lY3Rpb25zOiA4MAogIHRvbWNhdDoKICAgIGNvbm5lY3RvcjoKICAgICAgbWF4VGhyZWFkczogNTAKICAgICAgc2VuZFJlYXNvblBocmFzZTogZmFsc2UKICAgICAgZXh0cmFDb25maWc6IGFjY2VwdENvdW50PSIxMDAiCm1ldGFkYXRhOgogIGRhdGFiYXNlOgogICAgbWF4T3BlbkNvbm5lY3Rpb25zOiA4MApqZmNvbm5lY3Q6CiAgZW5hYmxlZDogdHJ1ZQo=
kind: Secret
metadata:
annotations:
meta.helm.sh/release-name: jfrt
meta.helm.sh/release-namespace: default
creationTimestamp: "2022-12-14T08:12:19Z"
labels:
app: artifactory
app.kubernetes.io/managed-by: Helm
chart: artifactory-107.47.12
heritage: Helm
release: jfrt
name: jfrt-artifactory-systemyaml
namespace: default
resourceVersion: "31547656"
uid: cbab4b27-0500-4e15-9f6a-171ca11c2238
type: Opaque
Here, in the above example, the dataKey.system.yaml content is in encoded format and we need to decode the content.
Content to be decoded from above example:
cm91dGVyOgogIHNlcnZpY2VSZWdpc3RyeToKICAgIGluc2VjdXJlOiBmYWxzZQpzaGFyZWQ6CiAgbG9nZ2luZzoKICAgIGNvbnNvbGVMb2c6CiAgICAgIGVuYWJsZWQ6IGZhbHNlCiAgZXh0cmFKYXZhT3B0czogPgogICAgLURhcnRpZmFjdG9yeS5hY2Nlc3MuY2xpZW50Lm1heC5jb25uZWN0aW9ucz01MAogIGRhdGFiYXNlOgogICAgdHlwZTogcG9zdGdyZXNxbAogICAgdXJsOiAiamRiYzpwb3N0Z3Jlc3FsOi8vamZydC1wb3N0Z3Jlc3FsOjU0MzIvYXJ0aWZhY3RvcnkiCiAgICBkcml2ZXI6IG9yZy5wb3N0Z3Jlc3FsLkRyaXZlcgogICAgdXNlcm5hbWU6ICJhcnRpZmFjdG9yeSIKYXJ0aWZhY3Rvcnk6CiAgZGF0YWJhc2U6CiAgICBtYXhPcGVuQ29ubmVjdGlvbnM6IDgwCiAgdG9tY2F0OgogICAgbWFpbnRlbmFuY2VDb25uZWN0b3I6CiAgICAgIHBvcnQ6IDgwOTEKICAgIGNvbm5lY3RvcjoKICAgICAgbWF4VGhyZWFkczogMjAwCiAgICAgIHNlbmRSZWFzb25QaHJhc2U6IGZhbHNlCiAgICAgIGV4dHJhQ29uZmlnOiBhY2NlcHRDb3VudD0iMTAwIgpmcm9udGVuZDoKICBzZXNzaW9uOgogICAgdGltZU1pbnV0ZXM6ICIzMCIKYWNjZXNzOgogIGRhdGFiYXNlOgogICAgbWF4T3BlbkNvbm5lY3Rpb25zOiA4MAogIHRvbWNhdDoKICAgIGNvbm5lY3RvcjoKICAgICAgbWF4VGhyZWFkczogNTAKICAgICAgc2VuZFJlYXNvblBocmFzZTogZmFsc2UKICAgICAgZXh0cmFDb25maWc6IGFjY2VwdENvdW50PSIxMDAiCm1ldGFkYXRhOgogIGRhdGFiYXNlOgogICAgbWF4T3BlbkNvbm5lY3Rpb25zOiA4MApqZmNvbm5lY3Q6CiAgZW5hYmxlZDogdHJ1ZQo=
Decoded content:
router:
serviceRegistry:
insecure: false
shared:
logging:
consoleLog:
enabled: false
extraJavaOpts: >
-Dartifactory.access.client.max.connections=50
database:
type: postgresql
url: "jdbc:postgresql://jfrt-postgresql:5432/artifactory"
driver: org.postgresql.Driver
username: "artifactory"
artifactory:
database:
maxOpenConnections: 80
tomcat:
maintenanceConnector:
port: 8091
connector:
maxThreads: 200
sendReasonPhrase: false
extraConfig: acceptCount="100"
frontend:
session:
timeMinutes: "30"
access:
database:
maxOpenConnections: 80
tomcat:
connector:
maxThreads: 50
sendReasonPhrase: false
extraConfig: acceptCount="100"
metadata:
database:
maxOpenConnections: 80
jfconnect:
enabled: true
Note: By default the secrets will be encoded and decoded using the base64 format.
Hence, we can prefer using any external encoding/decoding tools to convert the string available in the dataKey.system.yaml.
Once it is decoded, update the system.yaml content with the necessary changes and then encode the yaml file content again as suggested above.
Here, we have updated the system.yaml content with following values shown below:router:
serviceRegistry:
insecure: false
shared:
logging:
consoleLog:
enabled: false
extraJavaOpts: >
-Dartifactory.access.client.max.connections=250 -Dartifactory.async.corePoolSize=16
database:
type: postgresql
url: "jdbc:postgresql://jfrt-postgresql:5432/artifactory"
driver: org.postgresql.Driver
username: "artifactory"
artifactory:
database:
maxOpenConnections: 300
tomcat:
maintenanceConnector:
port: 8091
connector:
maxThreads: 1200
sendReasonPhrase: false
extraConfig: acceptCount="100"
frontend:
session:
timeMinutes: "30"
access:
database:
maxOpenConnections: 300
tomcat:
connector:
maxThreads: 250
sendReasonPhrase: false
extraConfig: acceptCount="100"
metadata:
database:
maxOpenConnections: 300
jfconnect:
enabled: true
We will encode the above updated system YAML file content again:cm91dGVyOgogIHNlcnZpY2VSZWdpc3RyeToKICAgIGluc2VjdXJlOiBmYWxzZQpzaGFyZWQ6CiAg
bG9nZ2luZzoKICAgIGNvbnNvbGVMb2c6CiAgICAgIGVuYWJsZWQ6IGZhbHNlCiAgZXh0cmFKYXZh
T3B0czogPgogICAgLURhcnRpZmFjdG9yeS5hY2Nlc3MuY2xpZW50Lm1heC5jb25uZWN0aW9ucz0y
NTAgLURhcnRpZmFjdG9yeS5hc3luYy5jb3JlUG9vbFNpemU9MTYKICBkYXRhYmFzZToKICAgIHR5
cGU6IHBvc3RncmVzcWwKICAgIHVybDogImpkYmM6cG9zdGdyZXNxbDovL2pmcnQtcG9zdGdyZXNx
bDo1NDMyL2FydGlmYWN0b3J5IgogICAgZHJpdmVyOiBvcmcucG9zdGdyZXNxbC5Ecml2ZXIKICAg
IHVzZXJuYW1lOiAiYXJ0aWZhY3RvcnkiCmFydGlmYWN0b3J5OgogIGRhdGFiYXNlOgogICAgbWF4
T3BlbkNvbm5lY3Rpb25zOiAzMDAKICB0b21jYXQ6CiAgICBtYWludGVuYW5jZUNvbm5lY3RvcjoK
ICAgICAgcG9ydDogODA5MQogICAgY29ubmVjdG9yOgogICAgICBtYXhUaHJlYWRzOiAxMjAwCiAg
ICAgIHNlbmRSZWFzb25QaHJhc2U6IGZhbHNlCiAgICAgIGV4dHJhQ29uZmlnOiBhY2NlcHRDb3Vu
dD0iMTAwIgpmcm9udGVuZDoKICBzZXNzaW9uOgogICAgdGltZU1pbnV0ZXM6ICIzMCIKYWNjZXNz
OgogIGRhdGFiYXNlOgogICAgbWF4T3BlbkNvbm5lY3Rpb25zOiAzMDAKICB0b21jYXQ6CiAgICBj
b25uZWN0b3I6CiAgICAgIG1heFRocmVhZHM6IDI1MAogICAgICBzZW5kUmVhc29uUGhyYXNlOiBm
YWxzZQogICAgICBleHRyYUNvbmZpZzogYWNjZXB0Q291bnQ9IjEwMCIKbWV0YWRhdGE6CiAgZGF0
YWJhc2U6CiAgICBtYXhPcGVuQ29ubmVjdGlvbnM6IDMwMApqZmNvbm5lY3Q6CiAgZW5hYmxlZDog
dHJ1ZQo=
As a next step, edit the secret (kubectl edit secret <secret name>) and replace the new encoded key under the dataKey.system.yaml, then save it (:wq).
Finally, restart the Artifactory pods for the changes to take effect.
