1. Overview

There are two types of users in kubernetes: service account and regular user. These two user types correspond to two usage scenarios.

The service account is provided to the pods running in the cluster, and when these pods want to communicate with the apiserver, it is the serviceaccount that is used for authentication and authorization. Service accounts are stored in the k8s cluster and are RBAC-based and can be bound to roles to have specific permissions for specific resources.

Ordinary users are used for authentication and authorization in non-pod scenarios. For example, key components like k8s: scheduler, kubelet and controller manager, including using kubectl to interact with k8s clusters.

2. Service accounts

2.1 Automation

Even if we don’t create a service account under namespace or bind any service account to the pod, the serviceAccount field of the pod will be set to default. We can look at the service account named default and it will correspond to a secret. secret contains the values ca.crt, namespace and token.

This entire process is accomplished by three components.

  • Service account admission controller
  • Token controller
  • Service account controller

Among other things, the service account controller is responsible for maintaining the default service account under each namespace. This way, when a new namespace is created, a default service account is automatically created. Even if the service account is deleted, it will be automatically recreated immediately.

The token controller is responsible for several things.

  • Detects the creation of a service account and creates the appropriate Secret to support API access.
  • Detects the deletion of a service account and removes all corresponding Token Secret for the service account.
  • Detect the addition of a Secret, ensure that the corresponding service account exists, and add a token to the Secret if needed.
  • Detects the deletion of a Secret and removes the reference from the corresponding service account if needed.

The reason why you need to create a token for the service account and generate a secret will be mentioned later.

Now we know the automation process for service accounts and their tokens. What is the role of the service account access controller? Let’s look at the pod creation process. Most of the time we don’t assign a service account to a pod. So what is the default service account associated with a pod after it has been successfully created?

Here is where the service account admission controller comes into play. It makes changes to the pod through the admission controller mechanism. When a pod is created or updated, the following actions are performed.

  • If the pod does not have a ServiceAccount setting, set its ServiceAccount to default.
  • Ensure that the ServiceAccount associated with the pod exists, otherwise the pod is rejected.
  • If the pod does not contain an ImagePullSecrets setting, then add the ImagePullSecrets information from ServiceAccount to the pod.
  • Add a volume containing a token for API access to the pod.
  • Add volumeSource mounted in /var/run/secrets/kubernetes.io/serviceaccount to each container under the pod.

2.2 Authentication

Suppose our pod has set up a service account and now wants to communicate with the apiserver, how does the apiserver authenticate that the service account is valid?

As we mentioned above, the token controller creates a secret for the service account, and the token in the secret is the authentication information for the service account, specifically through JWT. This token stores the namespace, name, UID and secret name of the service account. If you are interested in the contents of the token, you can decode the token value in base64 and paste it into here to see the contents of the token. Then you can also paste the private key and public key to verify the signature is correct. The details are as follows.

1
2
3
4
5
6
7
8

{
  "iss": "kubernetes/serviceaccount",
  "kubernetes.io/serviceaccount/namespace": "default",
  "kubernetes.io/serviceaccount/secret.name": "default-token-894m5",
  "kubernetes.io/serviceaccount/service-account.name": "default",
  "kubernetes.io/serviceaccount/service-account.uid": "df5a8a9c-14d4-44c7-a55f-0100f51fc848",
  "sub": "system:serviceaccount:default:default"
}

This token is signed with a private key unique to the service account, which is passed in via --service-account-private-key-file when the contrller-manager is started. Similarly, in order to verify this token in the apiserver, the corresponding public key needs to be passed in at apiserver startup with the parameter ---service-account-key-file.

2.3 Authorization

Once the authentication problem is solved, how does the apiserver know if the requested service account has permission to operate the current resource? In k8s, the common authorization policy is RBAC. By associating a service account with a role, we can give the service account permission to operate the specified resource.

3. General users

3.1 Authentication

Unlike service accounts, k8s itself does not store any information about normal users, so how does apiserver authenticate normal users?

When a k8s cluster is created, there is usually a root certificate that is responsible for issuing the other certificates needed in the cluster. So it is assumed that if a common user can provide a certificate issued by the root certificate, he is a legitimate user. Where the common name of the certificate is the user name and the orgnization is the user group. For example, the certificate information of controller-manager on our local cluster is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

$ cfssl certinfo -cert controller-manager.pem
{
  "subject": {
    "common_name": "system:kube-controller-manager",
    "names": [
      "system:kube-controller-manager"
    ]
  },
  "issuer": {
    "common_name": "kubernetes",
    "names": [
      "kubernetes"
    ]
  },
  "serial_number": "7884702304157281003",
  "not_before": "2020-10-09T05:51:52Z",
  "not_after": "2021-10-09T05:51:54Z",
  "sigalg": "SHA256WithRSA",
  "authority_key_id": "11:F5:D7:48:AE:2E:7F:59:DD:4C:C4:A8:97:D2:C0:21:98:C6:3A:A7",
  "subject_key_id": "",
  "pem": "-----BEGIN CERTIFICATE-----\nMIIDCDCCAfCgAwIBAgIIbWwW+H7/quswDQYJKoZIhvcNAQELBQAwFTETMBEGA1UE\nAxMKa3ViZXJuZXRlczAeFw0yMDEwMDkwNTUxNTJaFw0yMTEwMDkwNTUxNTRaMCkx\nJzAlBgNVBAMTHnN5c3RlbTprdWJlLWNvbnRyb2xsZXItbWFuYWdlcjCCASIwDQYJ\nKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMPCkXDAttbJHnoLuhGFPr/28ag8NoI5\nY0e00uv3ltyHlakfCeOV48eBgpMka3BdUxFOTHI5wtumlU3iymdDvTnKkLc75v6p\nQ0Hfx0DYz8ykDcHQ04hIsgyXecaHl+hfy90bYAbF8V43MjA0X2VmIyLxS6wXgeM6\n8d/jc1X8Ggpw53ow7L4XiCMlXDPwzLlVUThYHRN+PA5EdADZHAzgXjsyg379/ori\nbS/NZtmizzfHGWugrfwBGPL187mp1xN1tyjR+7obtsQYpgZ0Emz74fWNlike2I69\ntlBDWYC5ddsbHtDu4h/H5guwFtZ3+VVLogyw3CntPvoV840Ro5jxmtMCAwEAAaNI\nMEYwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoGCCsGAQUFBwMCMB8GA1UdIwQY\nMBaAFBH110iuLn9Z3UzEqJfSwCGYxjqnMA0GCSqGSIb3DQEBCwUAA4IBAQBvUxh0\n+TDJn19qJPWXu5MGrRs1Efn+KCgSVMDcak9MfnG3kzCZ94SKw5PRYGQ6fzuUsgwT\nkbGJ3o4PR/BkZ9R2UUHa2prydQTHN+Qb/DuF3kVYTRbWxTN3br8Tp1uqiQVOLPe0\nrfRelwVR6y39O5Wc3VQCnQKM/ih4k2LKGwinq2sO7HN6pjwoKfapaOb050vrGOTu\n5RmX+SWs7CeWzITjC3sLfFyP/lh8zK7TINOKRx1/QBHlCnX4wnsXpOIe4Jf4ol1b\nKKGcicAcSrj252oOIxspAW8a7vX4DjVGRTSneQen5wbHeZbkeMyuvAVs2a73x94d\nfTH4K9+zxCLAVZFs\n-----END CERTIFICATE-----\n"
}

Where subject is the information of the certificate applicant and issuer is the information of the issuer. Here we know that controller-manager is using the user system:kube-controller-manager.

3.2 Authorization

Like the authorization of service accounts, normal users can bind roles and have certain privileges on certain resources through the RBAC mechanism. For example, the ClusterRoleBinding information for controller-manager is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

$ kubectl get clusterrolebinding  system:kube-controller-manager -o yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  creationTimestamp: "2020-09-22T12:33:24Z"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: system:kube-controller-manager
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:kube-controller-manager
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: system:kube-controller-manager

That is, it is bound to the system:kube-controller-manager cluster role, and you can continue to see what resources are bound to this role with extreme operational privileges if you are interested.

3.3 Practice

Since normal users need to issue their own credentials for them and then authorize them. Here’s a simple example to walk through.

First create a json file for the certificate issuance request.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

{
    "CN": "jiang",
    "key": {
        "algo": "ecdsa",
        "size": 256
    },
    "names": [
        {
            "O": "dev"
        }
    ]
}

CN is short for common name. O is short for orgnization, which can be interpreted as a user group. The next step is to issue the root certificate with k8s, which requires ca.crt and ca.key, which can be found in /etc/kubernetes/certs on the master node or elsewhere.

1

cfssl gencert -ca ca.crt -ca-key=ca.key jiang.json | cfssljson -bare jiang

This command will generate three new files:

  • jiang-key.pem: private key
  • jiang.pem: certificate
  • jiang.csr: certificate issuance request file

We use jiang’s private key and certificate to generate the user in kubeconfig as follows.

1

kubectl config set-credentials jiang --client-key=./jiang-key.pem --client-certificate=./jiang.pem --embed-certs

Next, generate a new context, specifying the users to be used by the k8s cluster.

1

kubectl config set-context k8s-jiang --user=jiang --cluster=k8s

Create roles and bindings for the user jiang. Here only the user jiang is allowed to read the pods under default namespace.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
    name: pod-reader
    namespace: default
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
    name: read-pods
    namespace: default
subjects:
- kind: User
  name: jiang
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

This completes the authentication and authorization of the user jiang. Use the following command to switch to this user.

1

kubectl config use-context k8s-jiang