Helm client use private registry¶

In this tutorial, I will show how to use helm client to interact with a private chart registry. - Add the private repo to the helm client - Create a custom helm chart - publish the chart to the private repo - Deploy an instance by downloading the helm chart from the private repo.

In this tutorial, the private registry is built by using the chartmuseum.

1. Add the private registry to the helm client¶

In this example, the private registry uses the basic auth to authenticate users

# add the registry to helm client
# I named the repo as cm
helm repo add --username admin --password changeMe cm https://chart.casd.local/

# look up available charts in the repo
helm search repo cm

2. Create a custom helm chart¶

Now let's create a new chart with name mario. This chart will deploy the famous game mario as a web page game.

# below command creates a chart skeleton
# the name of the chart provided here (e.g. mario) 
# will be the directory's name where the chart is created and stored.
helm create mario

# check the content of the generated mario folder
tree -L 3 mario/

# the architecture of the generated skeleton
mario/
├── charts
├── Chart.yaml
├── templates
│   ├── deployment.yaml
│   ├── _helpers.tpl
│   ├── hpa.yaml
│   ├── ingress.yaml
│   ├── NOTES.txt
│   ├── serviceaccount.yaml
│   ├── service.yaml
│   └── tests
│       └── test-connection.yaml
└── values.yaml

Let's understand the relevance of these created files and folders:

charts: This is an optional directory that may contain sub-charts
Chart.yaml: This is the main file that contains the description of our chart
templates: This is the directory where Kubernetes resources are defined as templates
values.yaml: This is the file that contains the default config values for our chart
.helmignore: This is where we can define patterns to ignore when packaging (similar in concept to .gitignore)

Suppose we have an origin deployment, service and ingress configuration as shown below.

mario_deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mario
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mario
  template:
    metadata:
      labels:
        app: mario
    spec:
      containers:
        - name: mario
          image: reg.casd.local/casd/docker-supermario
          ports:
            - name: http
              containerPort: 8080
      imagePullSecrets:
        - name: harbor-auth

- mario_service.yaml

apiVersion: v1
kind: Service
metadata:
  name: mario
spec:
  type: ClusterIP
  ports:
    - name: http
      targetPort: 8080
      port: 80
  selector:
    app: mario

mario_ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-mario
spec:
  # tls:
  #   - hosts:
  #       - mario.kub.sspcloud.fr
  rules:
    - host: mario.casd.dev
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: mario
                port:
                  number: 80

In general, you need to at least modify the below files in the generated skeleton: - Chart.yaml: name, description of the app. Pay attention of the value of appVersion, it will be used in the template templates/deployment.yaml as default image tag if the image tag value is empty in values.yaml. - templates/deployment.yaml: You need to adapt the template based on the origin deployment.yaml. For example, the containerPort value depends on how the image is build. In general, the default value will not work. - templates/service.yaml: You need to adapt the template based on the origin service.yaml. For example, the targetPort value depends on how the deployment.yaml is specified. You can't put a value which does not match. - templates/ingress.yaml: You need to adapt the template based on the origin ingress.yaml. - values.yaml: This stores all default value of the chart. If user provide nothing, the deployed instance will use the value in this file.

You can find the complete chart example in src/k8s/helm/custom_chart/mario.

Once, the chart is published, the user only need to modify the values.yaml to deploy it on a k8s cluster. If user need to modify the template, it means the chart is not well-designed.

3.1 Validating a chart¶

Before you deploy your chart, it's recommended that you valid your chart first (well-formed). You can use helm lint to do that. Below command is an example

# validate the chart syntax
helm lint ./mario

You can also render the generated k8s resource with the given default values.yaml

helm template ./mario

3.2 Deploy a release with helm chart¶

Once we've verified the chart to be fine, finally, we can below command to install the chart into the Kubernetes cluster

helm install mario-test ./mario

# you can view the deployed release 
helm ls -n <name-space>

# upgrade your release with new chart version
helm upgrade mario-test ./mario

# you can notice that after each upgrade, the revision number increase, so you can rollback
# with any number that inferior than current. Below example will rollback to 1
helm rollback mario-test 1

# delete the release
helm uninstall mario-test

4. Publish your chart to private registry¶

The below steps only works for registries built by chartmuseum. There are two ways to push charts to ChartMuseum: - via the api of chartMuseum - via helm cm-push plugin, the easiest way is to use helm cm-push plugin. You can find the official github page here

4.1 Publish via the api¶

# package your chart source
cd mario/
helm package .

# this command will generate a .tgz file. The version comes from the version value of `Chart.yaml`
mario-0.1.0.tgz 

# upload the binary to the registry
# if you have setup an auto redirect from http to https, you must call https in the curl command, 
# otherwise you will get a 301 status error due to the redirection.
curl --data-binary "@mario-0.1.0.tgz" https://chart.casd.local/api/charts

# If you’ve signed your package and generated a provenance file, upload it with:
curl --data-binary "@mario-0.1.0.tgz.prov" https://chart.casd.local/api/prov

# upload the package and provenance file at same time
curl -F "chart=@mario-0.1.0.tgz" -F "prov=@mario-0.1.0.tgz.prov" https://chart.casd.local/api/charts

# check if a chart exists in the registry or not, this will return a list of all available charts
curl https://chart.casd.local/api/charts

# delete a chart
curl -X DELETE https://chart.casd.local/api/charts/mario/0.1.0

4.2 Publish via cm-push plugin¶

The cm-push plugin is not installed by default, you need to install it first.

# install the plugin
helm plugin install https://github.com/chartmuseum/helm-push

# check the installed plugin
helm cm-push  --help

# add your private chartmuseum as a repo
helm repo add --username admin --password changeMe cm https://chart.casd.local/

# push the chart, with the plugin, you don't need to do helm package anymore
# you can push the directory directly, the plugin will package the chart, then push
helm cm-push mario/ cm

# Push .tgz package is still supported
helm cm-push mario-0.1.0.tgz cm

# push with a custom version
helm cm-push mario/ --version="0.2.0" cm

# If your ChartMuseum install is configured with ALLOW_OVERWRITE=true, chart versions will be automatically overwritten upon re-upload.
# Otherwise, the upload will be denied with message file already exist. Unless your install is configured with DISABLE_FORCE_OVERWRITE=true (ChartMuseum > v0.7.1), you can use the --force/-f option to to force an upload to overwrite an existing chart
helm cm-push --force mario-0.2.1.tgz chartmuseum

# push without adding chart repo. Below example shows how to push to an repo directly
helm cm-push mario-0.2.1.tgz http://chart.casd.local/

To check if the register is updated or not. You can follow the below commands

# list all available repo
helm repo list

# update the index of a repo
helm repo update

# Search a chart on all the added repo with a give keyword
helm search repo <repo-name>

# if you want to use regex, you need to use option -r
helm search repo -r "*mario*"

# to further filter your result, you can add an grep after
helm search repo -r "nginx" | grep -i "bitnami"

5. Remove the private registry¶

# Remove a repo
helm repo remove <repo-name>