Manage Kubernetes manifest templates using the CUE language

CUE is an open source data validation language and inference engine with its roots in logic programming.

To be honest, I didn’t understand the description at first either. Let’s look at a small example.

txt

moscow: {
name:    "Moscow"
pop:     11.92M
capital: true
}

Schema

municipality: {
name:    string
pop:     int
capital: bool
}

CUE

largeCapital: {
name:    string
pop:     >5M
capital: true
}

When defining JSON data, we usually treat Data and Schema separately. CUE combines the two by specifying both the data field type and the specific value, i.e. CUE does not make a distinction between “type” and “value”, both string and "Moscow" will be Both string and "Moscow" will be treated as values, but there is an order of inclusion between them, where "Moscow" can be subsumed under string, and then string will take precedence over "Moscow" in the lattice.

More about CUE can be found in the official documentation and definitions, so I won’t go into it here.

Using CUE for template rendering

CUE has a lot of cool usage scenarios, and first let’s focus on the profile rendering capabilities within it.

We borrow here from KubeVela example in the documentation

template: |
  parameter: {
      domain: string
      http: [string]: int
  }

  // trait template can have multiple outputs in one trait
  outputs: service: {
      apiVersion: "v1"
      kind:       "Service"
      spec: {
          selector:
              app: context.name
          ports: [
              for k, v in parameter.http {
                  port:       v
                  targetPort: v
              },
          ]
      }
  }

  outputs: ingress: {
      apiVersion: "networking.k8s.io/v1beta1"
      kind:       "Ingress"
      metadata:
          name: context.name
      spec: {
          rules: [{
              host: parameter.domain
              http: {
                  paths: [
                      for k, v in parameter.http {
                          path: k
                          backend: {
                              serviceName: context.name
                              servicePort: v
                          }
                      },
                  ]
              }
          }]
      }
  }

As you can see, you only need to pass in parameter to get output containing Service and Ingress.

This way of writing immediately reminds us of a similar tool, Helm, an early CNCF graduation project that has slowly evolved into a de facto industry standard in the field of k8s configuration definition. So what are the similarities and differences between writing configuration file rendering with CUE compared to Helm?

CUE vs Helm

The most intuitive feeling is that CUE is much smoother than Helm in template writing, mainly because of the difference in their initial design ideas; Helm relies on the plain text rendering capabilities of Go and String, while for CUE, JSON is a first-class citizen. So CUE (which is also a JSON superset) has a natural advantage in rendering yaml (JSON superset) files like K8S.

Talk is cheap, show me the code.

Let’s look at a few common scenarios together.

Named Templates

Helm

# values.yaml
app_name: "example_code"
other_attr1: "foo"
other_attr2: "bar"

# labels.tpl
{{- define "foo.labels" -}}
label1: {{ .Values.app_name | required "app_name is required" }} 
label2: {{ .Values.other_attr2 }}
{{ include "foo.selectorLabels" . }}
{{- end }}

{{- define "foo.selectorLabels" -}}
label3: {{ printf "%s-%s" .Values.app_name .Values.other_attr1 }}
{{- end }}

CUE

# labels.cue
app_name: *"example_code" ｜ string
other_attr1: "foo"
other_attr2: "bar"

selector_labels: {
    label3: "\( app_name )-\( other_attr2 )"
}

labels: {
    label1: app_name
label2: other_attr2
    selector_labels
}

Helm can define reusable templates in .tpl files and support other templates to refer to it, and only the defined templates can be reused, so you need to define a lot of additional base templates in complex Chart projects.
Compared to Helm’s cumbersome writing style, all content in CUE is JSON objects, no additional syntax is needed to specify the template, and any JSON objects can be referenced to each other.

Blank lines and indentations

Helm

apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "foo.deploymentName" . }}
labels:
    {{- include "foo.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
    matchLabels:
    {{- include "foo.selectorLabels" . | nindent 6 }}
strategy:
    {{- include "foo.updateStrategy" . | nindent 4 }}
revisionHistoryLimit: {{ .Values.revisionHistoryLimit }}
template:
    metadata:
    {{- with .Values.podAnnotations }}
    annotations:
        {{- toYaml . | nindent 8 }}
    {{- end }}
    labels:
        {{- include "foo.labels" . | nindent 8 }}

CUE

deployment: {
    apiVersion: "apps/v1"
    kind:       "Deployment"
    metadata: {
        name:   deployment_name
        labels: labels
    }
    spec: {
        replicas: runtime.replicas
        selector: matchLabels: selector_labels
        strategy:             update_strategy
        revisionHistoryLimit: revision_history_limit

Helm has a lot of markup characters like {{- include }} and nindent that are irrelevant to the actual logic, requiring spaces and indentation to be counted at each reference.
Whereas in CUE there is much less useless coding, no need for too many {{ * }} to mark up blocks of code, much higher information density, and complete liberation in terms of indentation and spaces.

values.yaml self-referencing

A long-standing headache in Helm is the inability to gracefully implement the values.yaml referencing problem.

Let’s look at the following example.

rootDomain: ""

#  What we expect is splicing by reference to the rootDomain, e.g. "foo.{{ .Values.rootDomain }}"
productDomain: ""

In the usual case, Chart users need to fill in the content for both variables separately, increasing the possibility of errors.

Although we can achieve this by defining templates.

# values.yaml
rootDomain: ""
productDomain: "foo.{{ .Values.rootDomain }}"

# helpers.tpl
{{- define "foo.productDomain" -}}
{{ tpl .Values.productDomain $ }}
{{- end }}

# ingress.yaml
...
spec:
  rules:
    - host: {{ include "foo.productDomain" . | quote }}
...

However, in practice, all references require additional include and the maintenance of the definitions is very labor-intensive (always make sure that blank lines and indentation are not wrong).

In CUE, however, cross-referencing seems natural and comfortable.

1
2

rootDomain: string
productDomain: *"foo.\( rootDomain )" | string

Importing Kubernetes packages

Another big killer of CUE is the ability to generate description cue files against native Kubernetes source code, and all k8s resource-related configuration files can naturally have schema checks. A specific tutorial can be found at Kubernetes tutorial.

package templates

import (
    apps "k8s.io/api/apps/v1"
)

deployment: apps.#Deployment

deployment: {
    apiVersion: "apps/v1"
    kind:       "Deployment"
    metadata: {
        name:   deployment_name
        labels: _labels
    }
...

In an example like this one, the deployment we define will be checked by apps.#Deployment, easily detecting fields that are not legal. In theory, you could generate cue files for multiple versions of k8s resources and use them all as model definitions for deployment, allowing for multi-version compatibility of resource definitions.

With all the benefits mentioned, replace all Helm Chart with CUE now?

Not yet, not yet.

Because Helm is a Package Manager that has some application management features such as Helm install, Helm rollback, etc. in addition to Chart rendering, while CUE is just a template. So in some cases where we only use Helm’s templates, we can consider migrating to CUE, but in other cases, we should use Helm.

Table of Contents