Helm is a highly recommended tool for managing K8S clusters, as it provides a clearer and more efficient way to manage K8S resources. Here is a summary of how Helm has been used in real life projects. Before you start using Helm tools, install the tools and configure the environment in advance.

Helm is very easy to install, and if the local kubectl can access the K8S cluster properly, then Helm is ready to use.

It is important to note that Helm version 3.0 and above is installed. This is because after 3.0 Helm no longer requires the tiller server to be installed in the cluster and the installation history can be recorded directly in the ConfigMap resource.

Infrastructure

Before you can start deploying a K8S cluster, the first thing you may need is to deploy the infrastructure for the entire cluster. For example, database services, caching services, queuing services, etc.

Build your own infrastructure

If you need to build your own infrastructure, i.e. build the base services directly within the cluster, without using external services. The normal installation process is as follows.

  • Query for available resources

    1

    $: helm search repo mysql

    If you can’t find the appropriate resource, you can just Google the corresponding resource. The so-called repo is actually a simple index of Helm Charts.

  • Adding a resource repository

    1

    $: helm repo add jaegertracing https://jaegertracing.github.io/helm-charts

  • View local resource repository

    1

    $: helm repo list

    Once you have found the resources you need, you will need to check yourself if the default configuration of the resources meets the requirements before deploying them, and customise the settings if they do not. Once you have completed the customisation, you can then deploy it online. The process is as follows.

    1
2
3
4
5
6
7
8

    # Pull mysql helm charts to local directory
$: helm pull bitnami/mysql --untar

# Just change the definition configuration
$: vim mysql/values.yaml

# Update or install
$: helm upgrade --install --namespace=[YOUR NAMESPACE] --atomic --history-max 3 mysql-demo ./mysql/ --values ./mysql/values.yaml

External infrastructure

If you do not need to build your own infrastructure within the cluster, but use external resources directly. It is easier to configure the external infrastructure within the cluster by simply defining the appropriate service resources.

However, to make it easier to manage, the installation of the external infrastructure can also be managed via helm. The basic process is as follows.

1
2

# Create a new Helm Charts, responsible for managing all external resources
$: helm create external-services

The Helm Chart, created by default, creates a number of functional templates that can be selected as required. For external infrastructures, you only need to define the corresponding service resources, and all other unnecessary resource templates can be deleted directly. Take Ali RDS resources for example, delete the irrelevant resource templates under external-services/templates. Just define a mysql.yaml template.

1
2
3
4
5
6
7
8

apiVersion: v1
kind: Service
metadata:
  name: mysql  
spec:
  externalName: {{.Values.mysql.address}}
  sessionAffinity: None
  type: ExternalName

Clear the external-services/values.yaml irrelevant configuration and just provide a mysql.address setting.

Cluster Configuration

Once the infrastructure of the cluster has been configured, it is time to complete the configuration resources for the entire cluster. The configuration resources, again, are managed via a dedicated Helm charts.

1
2

# Create a new Helm Charts, responsible for managing the cluster configuration
$: helm create settings

Remove extraneous resource templates. Start adding cluster configurations. Generally speaking, cluster configurations are also of the following configuration types:

  • Certificate type configuration
  • Password-based configuration
  • Container image access authentication configuration
  • Generic business service configuration

The above configurations can be configured in a K8S cluster via the Secret resource and the ConfigMap resource.

Certificate configuration

The main purpose of certificate configuration is to automate the configuration of certificate files into the cluster, which can be read through the Helm template function .Files.Get. Take a common domain name certificate as an example:

1
2
3
4
5
6
7
8

apiVersion: v1
data:
  tls.crt: "{{ .Files.Get "files/example.com.pem" | b64enc }}"
  tls.key: "{{ .Files.Get "files/example.com.key" | b64enc }}"
kind: Secret
metadata:
  name: www-example-com-tls
type: IngressTLS

Simply store the certificate file in the settings/files subdirectory.

Password configuration

Password configuration is again very simple, e.g:

1
2
3
4
5
6
7
8

apiVersion: v1
data:
  ACCESS: {{ .Values.demo.access | b64enc }}
  SECRET: {{ .Values.demo.secret | b64enc }}
kind: Secret
metadata:
  name: demo-secret
type: Opaque

This is done by setting the specific password information in settings/values.yaml.

Container image access authentication configuration

In order for the cluster to obtain the corresponding service container image when publishing services, it is also necessary to provide authentication information for the service container image server. Normally, the authentication information for the image server can be created via the command line. If storage is required, the authentication information resource created on the command line can be downloaded and saved locally.

1
2

# Authentication information on how to create a image service can be found via the command help
$: kubectl create secret docker-registry -h

Generic service configuration

Generic service configurations, usually shared by a group of business services and not very sensitive, can be configured in the form of ConfigMap resources.

1
2
3
4
5
6
7
8

apiVersion: v1
kind: ConfigMap
metadata:
  name: your-config
data:
  [yourKey1]: {{.values.yourValue1}}
  [yourKey2]: {{.values.yourValue2}}
  [yourKey3]: {{.values.yourValue3}}

It’s very simple.

Once you have configured all of the above, it is very clear that you can manage them all directly in settings.

Business Services

Unlike configuration, business services are similar to self-built infrastructure services and typically require the provisioning of deployment resources that may include

  • deployment or statefulset
  • service
  • ingress
  • configmap for your own configuration only
  • secret for own keys only

So, there will be more types of resource definitions for business services. A brief list of the configurations for regular services:

  • StatefulSet

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

    apiVersion: apps/v1
kind: StatefulSet
metadata:
name: demo
labels:
    app: demo
spec:
serviceName: demo
replicas: {{ .Values.deployment.replicaCount }}
selector:
    matchLabels:
    app: demo
template:
    metadata:
    annotations:
        timstamp: "{{ date "20060102150405" .Release.Time }}" 
    labels:
        app: demo
    spec:
    {{- with .Values.imagePullSecrets }}
    imagePullSecrets:
            {{- toYaml . | nindent 8 }}
    {{- end }}
    containers:
        - name: demo
        env:
            - name: DEMO_CONFIG
            valueFrom:
                configMapKeyRef:
                key: CONFIG
                name: demo-config
            - name: DEMO_SECRET
            valueFrom:
                secretKeyRef:
                key: SECRET
                name: demo-secret
        ports:
            - name: http
            containerPort: 8080
        image: "{{ .Values.deployment.image.name }}:{{ .Values.deployment.image.tag }}"
        imagePullPolicy: {{ .Values.deployment.image.imagePullPolicy }}
        command: ["demo", "http"]

    The values.yaml configuration is provided.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10

    deployment:
replicaCount: 1
image:
    name: "your.image.registry/group/image.name"
    tag: latest
    imagePullPolicy: Always

imagePullSecrets:
- name: "image-pull-secret-1"
- name: "image-pull-secret-2"

  • Service configuration

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

    apiVersion: v1
kind: Service
metadata:
name: demo
labels:
    app: demo
spec:
type: ClusterIP
ports:
    - name: demo
    port: 80
    targetPort: http
selector:
    app: demo

  • Ingress Configuration

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: demo
spec:
tls:
    - hosts:
        - {{.Values.ingress.host.name}}
    secretName: {{.Values.ingress.tls.secretName}}
rules:
    - host: {{.Values.ingress.host.name}}
    http:
        paths:
        - pathType: Prefix
            path: /v1/api/demo
            backend:
            service:
                name: demo
                port:
                number: 80

Summary

This is how you can manage the classification and deployment of K8S general resources through Helm. If you want to further standardise the management of resources, you can also build your own Helm Charts resource indexing service. The Helm Chart for all business services can be packaged and managed. The command is helm package.