client-go is the programmatic interactive client library used by the Kubernetes project. The core source code related to resource interaction is separated out as a project, client-go. That is to say, the Kubernetes used now is integrated with client-go, so the coding quality of this library should be assured.

client-go is a programmatic interactive client library that allows you to add, delete, modify, and query resource objects in a kubernetes cluster by writing Go code.

Resource objects include deployment, service, ingress, replicaSet, pod, namespace, node, etc.

Source Code Introduction

Brief description of the source code directory

  • discovery: service discovery via the Kubernetes API.
  • kubernetes: provides ClientSet clients that can operate on Kubernetes built-in resource objects.
  • dynamic: provides DynamicClient client that enables operations on any Kubernetes resource object.
  • rest: provides RESTClient clients that can perform REST requests to kube-apiserver for resource operations.
  • scale: provides the ScaleClient client, which is mainly used for scaling up and down resources such as Deployment.
  • listers: provides Lister functionality for Kubernetes resources, providing read-only cached data for Get / List requests.
  • informers: Provides Informer implementations for each Kubernetes resource.
  • transport: for providing secure TCP connections.
  • tools/cache: provides common tools; provides Client query and caching mechanisms to relieve kube-apiserver pressure.
  • util: provides common methods.

Client Objects

A large part of learning client-go for kubernetes secondary development is learning to become proficient with its several clients. client-go has the following four client client objects that connect to the kube-apiserver of a given cluster via kubeconfig configuration information to enable resource-related operations The kubeconfig configuration information is used to connect to the kube-apiserver of a given cluster to perform resource-related operations.

  • RESTClient: the most basic client in client-go, all other clients are implemented based on RESTClient, which implements RESTful style API request encapsulation and can realize RESTful style interaction with any Kubernetes resources (including built-in resources and CRDs). RESTful-style interactions with any Kubernetes resource (including built-in resources and CRDs), such as Post() / Delete() / Put() / Get(), with support for Json and protobuf.
  • ClientSet: the most commonly used Client for interacting with Kubernetes built-in resource objects, emphasizing that it can only handle Kubernetes built-in resources, not CRD custom resources, and when using it, you need to specify the Group, specify the Version, and then get it according to the Resource. clientSet The operation code of ClientSet is automatically generated by the client-gen code generator.
  • DynamicClient: DynamicClient can handle any kubernetes resource, including CRD custom resources. However, note that the object returned by DynamicClient is a map[string]interface{}. If you need to control all APIs in a controller, you can use dynamic client, which only supports JSON.
  • DiscoveryClient: used to discover Group / Version / Resource information supported by kube-apiserver.

client-go client initialization

kubeconfig configuration information

If client-go wants to access the kube-apiserver for interactive operations, it must first configure the connection and authentication information. The configuration information is provided by the kubeconfig file. By default, the kubeconfig file for the cluster is stored in the following path.

1

$HOME/.kube/config

Take the kubeconfig file of my test cluster as an example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

apiVersion: v1
kind: Config
clusters:
  - cluster:
      name: kubernetes
      server: https://node01:6443
      certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJ...
contexts:
  - context:
      cluster: kubernetes
      user: kubernetes-admin
      name: kubernetes-admin@kubernetes
      current-context: kubernetes-admin@kubernetes
users:
  - name: kubernetes-admin
    user:
      client-certificate-data: LS0tLS1CRUdJTiBDM4akND...
      client-key-data: LS0tLS1CFURSBLRVktLS0tLQpNS...

It mainly contains the following information.

  • apiVersion: the version of the configuration file resource
  • kind: the type of the configuration file resource, i.e. Config
  • clusters: defines the information about Kubernetes clusters
    • clusters: defines the name, kube-apiserver address, and certificate information for each cluster.
  • contexts: cluster context environment
    • context: defines the command space and user information for each specific cluster, which is used to send requests to the specified cluster.
  • users: defines user authentication information, client credentials.

client-go reads kubeconfig configuration information to generate config objects.

1
2
3

...
config, err := clientcmd.BuildConfigFromFlags("","./configs/kubeconfig.conf")
...

1
2

kubeconfig, _ = ioutil.ReadFile("./configs/kubeconfig.conf")
restConf, _ = clientcmd.RESTConfigFromKubeConfig(kubeconfig)

The config object further generates the required client object to communicate with the kube-apiserver for resource interaction. Each type of client object has its own generation method.

Common Client initialization and resource manipulation examples

We will introduce the usage of RESTClient / ClientSet / DynamicClient, and we will not introduce DiscoveryClient.

RESTClient

RESTClient initialization

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

config, err := clientcmd.BuildConfigFromFlags("","./configs/kubeconfig.conf")
if err != nil{
  panic(err)
}

config.APIPath = "api"
config.GroupVersion = &corev1.SchemeGroupVersion
config.NegotiatedSerializer = scheme.Codecs

restClient, err := rest.RESTClient(config)
if err != nil{
  panic(err)
}

RESTClient Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

pods := &corev1.PodList{}
err = restClient.Get()
                .Namespace("yingchi")
                .Resource("pods")
                        .VersionedParams(&metav1.ListOptions{}, scheme.ParameterCodec)
                .Do()
                .Into(pods)
if err != nil {
  panic(err)
}

ClientSet

ClientSet is the most commonly used client object in client-go for the reason that it is much easier to use than RESTClient. In particular, ClientSet should be the preferred Client object when just manipulating Kubernetes built-in resources.

ClientSet encapsulates the management of Version and Resource on top of RESTClient, and each Resource can be understood as a client, which is how ClientSet got its name.

ClientSet initialization

1
2
3
4
5
6
7
8
9

config, err := clientcmd.BuildConfigFromFlags("","./configs/kubeconfig.conf")
if err != nil{
  panic(err)
}

clientSet, err := kubernetes.NewForConfig(config)
if err != nil{
  panic(err)
}

ClientSet example

1
2
3
4
5
6

pods, err := clientSet.CoreV1()
                      .Pods("yingchi")
                      .List(metav1.ListOptions{})
if err != nil{
  panic(err)
}

DynamicClient

DynamicClient is also a dynamic client based on the RESTClient wrapper. Unlike ClientSet, DynamicClient can handle any kubernetes resource, including CRD custom resources.

The reason it can handle CRDs is that DynamicClient internally implements Unstructured to handle unstructured or unknown structured data structures, which is the key to handling CRD type resources, while ClientSet internally has structured data, i.e., it knows the specific resource corresponding to each Resource and Version data type corresponding to each Resource and Version.

It should be noted that DynamicClient is not type-safe, so pay particular attention to pointers when interacting with resources using DynamicClient, as improper handling may cause the program to crash.

DynamicClient initialization

1
2
3
4
5
6
7
8
9

config, err := clientcmd.BuildConfigFromFlags("", "./configs/kubeconfig.conf")
if err != nil{
  panic(err)
}

dynamicClient, err := dynamic.NewForConfig(config)
if err != nil{
  panic(err)
}

DynamicClient Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

resource := schema.GroupVersionResource{
  Version: "v1",
  Resource: "pods"
}

obj, err := dynamicClient.Resource(resource)
                        .Namespace("yingchi")
                        .List(metav1.ListOptions{})
if err != nil{
  panic(err)
}

pods := &corev1.PodList{}
err = runtime.DefaultUnstructuredConverter
            .FromUnstructured(obj.UnstructuredContent(), pods)

Informer mechanism

A relatively high-end design in the Client-go package is the Informer design, which is used if you want a more elegant pattern of resource interaction.

We know from the previous example that we can interact directly through the Kubernetes API, but considering the form of interaction, the Informer is designed as a List/Watch approach, where the Informer first retrieves all object objects of the resource from the Kubernetes API through a List when it is initialized, and caches them at the same time. The list is then used to monitor and control the resource.

Then it monitors the resources through the Watch mechanism. This way, with the Informer and its cache, we can interact directly with the Informer instead of interacting with the Kubernetes API every time. This avoids the access pressure on the apiserver caused by polling the kube-apiserver, for example.

Informer provides an event handler mechanism and triggers callbacks so that upper layer applications such as Controller can handle specific business logic based on asynchronous callbacks.

Because Informer can monitor all events of all resources through the List, Watch mechanism. So just add the callback function instance of ResourceEventHandler instance to Informe r to implement OnAdd(obj interface{}) OnUpdate(oldObj, newObj interface{}) and OnDelete(obj interface{ }) are three methods that handle the creation, update and deletion of resources.

Informer is used by various controllers in Kubernetes.

Informer is the quintessence of client-go.