Power Up Helm Charts: Using Kustomize to Manage Kubernetes Deployments
Operations teams need to automate deployment of applications, and Helm is the industry-standard solution for deploying to Kubernetes. Like many software vendors, we provide Helm charts for installing Artifactory and other JFrog DevOps platform products, designed for the standard/recommended configurations most teams will need. These templates provide customers a limited set of configuration options.
If you need to customize your Artifactory, Xray, or other deployments in ways of your own, you might fork the chart to create your own custom version. But once you do, each time JFrog updates its Helm chart, your custom version becomes out of sync and obsolete. To keep your version up-to-date, you would need to reintegrate for every update.
How can you customize a Helm chart without forking? That’s exactly what Google created Kustomize for. In this post we’ll show you how you can use Kustomize overlays to perform custom deployments while always using the latest Helm chart version from your vendor.
Templates vs Overlays
A template is a form that has placeholders that an automated process will parse to replace them with values. Designed to perform a specific function, it marks the places where you must provide the specifics. If you’ve ever played “Mad Libs” this “fill-in-the-blanks” process will be familiar.
As a developer, you’ll recognize that templates and values are like macros and their variables, or subroutines and their parameters.
An overlay is a set of replacement strings. Blocks of text in the original file are entirely replaced with new blocks of text.
What’s the difference?
- A template needs to be carefully prepared to demand specific information in key places. When you use a template, you’re restricted to changing only those elements the template makes available.
- An overlay doesn’t require the original file to be prepared in any way. You can replace any part in its entirety.
HelloWorld Helm Chart
Since Artifactory’s Helm charts are quite advanced, let’s use a very simple example. Here, the template permits an argument for the company name.
$ cat templates/pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: helloworld
spec:
restartPolicy: Never
containers:
- name: hello
image: alpine
env:
command: ["/bin/sh","-c"]
args: ["/bin/echo Hello! My company name is {{ .Values.companyName}}"]
The values for the template’s arguments are in the file values.yaml.
$ cat values.yaml
companyName: ABC Company
Let’s install and see how it works.
$ Helm install helloworld .
NAME: helloworld
LAST DEPLOYED: Mon May 18 16:53:14 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
$ kubectl logs -f helloworld
Hello! My company name is ABC Company
Neat! This worked well, and we shipped it to customers. But one customer has a unique requirement, to add employee name and department name to the output to look like this:
My name is John. I work for the Accounting department. Our company name is ABC Company.
Okay, pretty easy, they could fork the Helm chart and change the Helm template as follows:
args: ["/bin/echo My name is {{ .Values.employeeName}}. I work for {{ .Values.employeeDepartment}} department. Our company name is {{.Values.companyName}}"]
Then they would add new values in the values.yaml file.
$ cat values.yaml
employeeName: Gary
employeeDepartment: Marketing
companyName: ABC Company
But, as we noted, their fork is now out of sync with the original. This is where Kustomize comes to the rescue.
Overlaying with Kustomize
Kustomize enables you to overlay your own “kustomizations” in yaml files. In our example, the customer can “kustomize” to their needs without creating a private, unmaintainable fork of charts.
First the customer will create a kustomization.yaml file.
patchesJson6902:
- target:
version: v1
kind: Pod
name: helloworld
patch: |-
- op: replace
path: /spec/containers/0/args
value: ["/bin/echo My name is {{ .Values.employeeName}}. I work for {{ .Values.employeeDepartment}} department. Our company name is {{ .Values.companyName}}"]
resources:
- templates/pod.yaml
Now we can direct kustomize to apply our overlay.
$ mkdir templates_new
$ kustomize build -o templates_new
$ cat templates_new/~g_v1_pod_helloworld.yaml
apiVersion: v1
kind: Pod
metadata:
name: helloworld
spec:
containers:
- args:
- /bin/echo My name is {{ .Values.employeeName}}. I work for {{ .Values.employeeDepartment}}
department. Our company name is {{ .Values.companyName}}
command:
- /bin/sh
- -c
image: alpine
name: hello
restartPolicy: Never
We will first replace the original template with our new template, then install it with Helm and verify.
$ mv templates templates_old
$ mv templates_new/ templates
$ Helm delete helloworld
release "helloworld" uninstalled
$ Helm install helloworld .
NAME: helloworld
LAST DEPLOYED: Tue May 19 14:27:18 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
$ kubectl logs -f helloworld
My name is Gary. I work for Marketing department. Our company name is ABC Company
Great! This matches our customer’s requirement.
Template First, Overlay Next
In the prior example, we used Kustomize to modify our Helm template to accept new values, then used that version with helm install to deploy the app.
But there are scenarios where that sequence isn’t ideal. Instead, you may wish to render the chart template first locally, then apply your Kustomize overlay when you deploy the app.
This may work best when you need to deploy the same app to multiple environments, but with cross-cutting concerns such as labels, security, or metering. For example you may have different combinations of requirements for each environment:
| Environment | labels | security | metering |
| dev | Yes | No | No |
| test | Yes | No | Yes |
| production | Yes | Yes | Yes |
Similarly, you may need to adjust ports or access for each of these environments. In these scenarios, it may be more flexible to apply a different Kustomize overlay to the same rendered Helm chart for each environment.
For this purpose, the kubectl command line utility provides the apply -k option. This facility applies Kustomize on charts using kustomization.yaml files in the directories.
To start, let’s render the Helm chart locally using the helm template command. This outputs a YAML file with all values resolved, which we capture to a local file.
$ mkdir templates_new
$ helm template . > templates_new/pod.yaml
$ $ cat templates_new/pod.yaml
---
# Source: helloworld/templates/~g_v1_pod_helloworld.yaml
apiVersion: v1
kind: Pod
metadata:
name: helloworld
spec:
containers:
- args:
- /bin/echo My name is Gary. I work for Marketing
department. Our company name is ABC Company
command:
- /bin/sh
- -c
env: null
image: alpine
name: hello
restartPolicy: Never
Create a new kustomization file to add labels to our pod as follows:
$ cat templates_new/kustomization.yaml
commonLabels:
app: helloworld
resources:
- templates_new/pod.yaml
Now let’s use kubectl apply -k to install our chart with new labels:
$ helm delete helloworld
$ kubectl apply -k templates_new/.
pod/helloworld created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
helloworld 0/1 Completed 0 10s
$ kubectl describe pod helloworld
Name: helloworld
Namespace: default
Priority: 0
Node: docker-desktop/192.168.65.3
Start Time: Mon, 22 Jun 2020 16:22:11 -0700
Labels: app=helloworld
Annotations: Status: Succeeded
...
...
Success! Now we are able to take any vendor’s Helm chart and add our own‘kustomizations on top while continuing to take updates from upstream charts.
Dive Deeper
JFrog’s Helm charts are quite advanced, and we’ve often had to maintain multiple versions for our internal use. Using Kustomize has helped us to avoid manual editing of charts, and to more fully automate our CI/CD processes.
Want to learn more? Watch the recording of our webinar on how to Power up Artifactory with Kustomize & Helm. We’re eager to share what we’ve learned and help you to better manage your deployments!
