Skip to content

Setup local helm chart registry

A local helm chart registry is essential when your k8s cluster does not have internet connection, or it can't use public helm chart repo to pull the helm chart.

In this tutorial, we will introduce two ways to set up a private helm chart registry - Harbor oci registry - chart museum

1. Use Harbor oci registry

Harbor provides an oci registry which can store, share helm charts.

If you want to play with the oci registry in Harbor, you can read this Harbor_helm_chart_management.md

But Onyxia does not support OCI registry. So we can not use harbor to manage helm chart.

2.Use chart museum

chartmuseum is an open-source Helm Chart Repository server written in Go (Golang), with support for various cloud storage backends.

2.1 Install Chart museum

You can find the official installation doc here. In this tutorial, I only focus on linux bare metal installation

# get the installation script and run it
curl https://raw.githubusercontent.com/helm/chartmuseum/main/scripts/get-chartmuseum | bash

# you should see below output, it means the chartmuseum binary is installed in /usr/local/bin
Downloading https://get.helm.sh/chartmuseum-v0.15.0-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install chartmuseum into /usr/local/bin
chartmuseum installed into /usr/local/bin/chartmuseum

# check the version
chartmuseum --version

# get help
chartmuseum --help

2.2 Configure chartmuseum

There are three ways to configure chartmuseum - command line options - env var - config file

Here we use config file, because it's simpler to communicate how the chartmuseum is built.

The options that can be used in the config file can be found in this file

Below is a simple config.yaml to run a minimum instance for test. In linux os-system, it's recommended to but the config.yaml file in /etc/chartmuseum.

debug: true
port: 8080
storage.backend: local
storage.local.rootdir: /data
basicauth.user: admin
basicauth.pass: changeMe
authanonymousget: true
depth: 0

# run the chartmuseum with the given config file
chartmuseum --config /etc/chartmuseum/config.yaml

# you can access the web interface
http://ip:8080

You can use Nginx as the reverse proxy to protect the chart museum. For more information, please visit this doc

2.3 Upload chart to chartMuseum

There are two ways to push charts to ChartMuseum: - via the api of chartMuseum - via helm cm-push plugin, the easiest way is to use helm cm-push plugin. You can find the official github page here

2.3.1 Upload chart via the api of chartMuseum

# push a chart via the api of chartMuseum
curl -F "chart=@hello-world-0.1.0.tgz" https://chart.casd.local/api/charts

curl --data-binary "@hello-world-0.1.0.tgz" https://chart.casd.local/api/charts

# If you’ve signed your package and generated a provenance file, upload it with:
curl --data-binary "@hello-world-0.1.0.tgz.prov" http://chart.casd.local/api/prov

# Or you can upload both at same time
curl -F "chart=@hello-world-0.1.0.tgz" -F "prov=@hello-world-0.1.0.tgz.prov" http://chart.casd.local/api/charts

The name of the .tgz will not impact the version of the chart, the chartMuseum will read the Chart.yaml in the package to determine version.

2.3.2 Upload chart via helm

# install the binary of helm push plugin
helm plugin install https://github.com/chartmuseum/helm-push

# check the installed plugin
helm cm-push  --help

# add your private chartmuseum as a repo
helm repo add --username admin --password changeMe cm https://chart.casd.local/

# list all added repo
helm repo list

# update the index of a repo
helm repo update

# Search a chart on all the added repo with a give keyword
helm search repo <keyword>

# if you want to use regex, you need to use option -r
helm search repo -r ".*"

# to further filter your result, you can add an grep after
helm search repo -r "nginx" | grep -i "bitnami"

# push the chart, with the plugin, you don't need to do helm package anymore
# you can push the directory directly, the plugin will package the chart, then push
helm cm-push hello-world/ cm

# Push .tgz package is still supported
helm cm-push hello-world-0.1.0.tgz cm

# push with a custom version
helm cm-push hello-world/ --version="0.2.0" cm

# If your ChartMuseum install is configured with ALLOW_OVERWRITE=true, chart versions will be automatically overwritten upon re-upload.
# Otherwise, the upload will be denied with message file already exist. Unless your install is configured with DISABLE_FORCE_OVERWRITE=true (ChartMuseum > v0.7.1), you can use the --force/-f option to to force an upload to overwrite an existing chart
helm cm-push --force hello-world-0.2.1.tgz chartmuseum

# push without adding chart repo. Below example shows how to push to an repo directly
helm cm-push hello-world-0.2.1.tgz http://chart.casd.local/


# Remove a repo
helm repo remove <repo-name>

note you need to run helm repo update to fetch the new index.yaml of each repo to get the latest uploaded chart

2.4 chartMuseum Authentication

Basic auth

If the chartMuseum is installed with basic authentication enabled, you need to add user credential when you add repo

# option 1
helm repo add --username admin --password changeMe cm https://chart.casd.local/

# option 2
# The plugin will use the auth info located in ~/.config/helm/repositories.yaml

# option 3
# Use below env var
export HELM_REPO_USERNAME="myuser"
export HELM_REPO_PASSWORD="mypass"

TLS

ChartMuseum uses the linux system ca-cert folder. If you use a self signed certificat, you can add the custom CA certificat on the server where you have installed the helm cm-push plugin.

If you don't have admin rights to do so, you can use below option to make changes on the plugin when adding repo

  • --ca-file string: Verify certificates of HTTPS-enabled servers using this CA bundle [$HELM_REPO_CA_FILE]
  • --cert-file string: Identify HTTPS client using this SSL certificate file [$HELM_REPO_CERT_FILE]
  • --key-file string: Identify HTTPS client using this SSL key file [$HELM_REPO_KEY_FILE]
  • --insecure: Connect to server with an insecure way by skipping certificate verification [$HELM_REPO_INSECURE]

Appendix

Set up a systemd daemon

To be able to run chartmuseum as a daemon, you can add the following file /etc/systemd/system/chartmuseum.service We recommend you to use the config.yaml to configure the chartmuseum daemon.

[Unit]
Description=chartmuseum
Documentation=Helm Chart Repository
Requires=network-online.target
After=network.target

[Service]
User=root
Restart=allways
ExecStart=/usr/local/bin/chartmuseum --config /etc/chartmuseum/config.yaml
ExecStop=/usr/local/bin/chartmuseum step-down
[Install]
WantedBy=multi-user.target

There is another way if you don't want to use the config.yaml, you can use $ARGS to specify the configuration. (Not recommended)

[Unit]
Description=chartmuseum
Documentation=Helm Chart Repository
Requires=network-online.target
After=network-online.target

[Service]
EnvironmentFile=/etc/chartmuseum/chartmuseum.config
User=root
Restart=allways
ExecStart=/usr/local/bin/chartmuseum $ARGS
ExecStop=/usr/local/bin/chartmuseum step-down

[Install]
WantedBy=multi-user.target

The chartmuseum.config looks like

ARGS=\
--port=8080 \
--storage="local" \
--storage-local-rootdir="/data" \
--log-json \
--basic-auth-user=admin \
--basic-auth-pass="changeMe" \
--auth-anonymous-get

How to generate a provenance file ?