Pod Security Policies

What is PodSecurityPolicy

PodSecurityPolicy is a global resource used to control Pod security-related configuration.

On a kubernetes cluster with RBAC enabled, if users are allowed to use kubectl, then PodSecurityPolicy must be enabled, otherwise users may use some privileged resources (e.g. privileged, hostNetwork, hostPath, etc.) and affect the stability of the node machine.

With PSP turned on, users can only use resources allowed by the administrator. PSP supports the following (see official website for details).

Enabling PodSecurityPolicy

Enabling is simple, just configure apiserver to add the admission plugin PodSecurityPolicy.

1

--enable-admission-plugins=NodeRestriction,PodSecurityPolicy

The apiserver will restart after the modification. Since there is no Policy configured here, PSP will prevent all Pod creation after reboot.

Since the PSP API policy/v1beta1/podsecuritypolicy is independent from the authorization controller, for existing clusters, it is recommended to configure psp and authorization first, and then turn on PS.

Authorization Policy

Policy itself does not produce the actual effect, you need to bind it to the user or serviceaccount to complete the authorization.

Binding method: Create a Role, which can use the PodSecurityPolicy, and then bind the role to the user or serviceaccount.

Example

Here is an example.

1. Create Policy

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: example
spec:
  privileged: false  # Don't allow privileged pods!
  # The rest fills in some required fields.
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  volumes:
  - '*'

2. Create serviceaccount

1
2
3
4

kubectl create namespace hellobaby
kubectl create sa fake-user
alias kubectl-admin='kubectl -n hellobaby'
alias kubectl-user='kubectl --as=system:serviceaccount:hellobaby:fake-user -n hellobaby'

3. Create Pod

1
2
3
4
5
6
7
8

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - image: nginx
    name: nginx

At this point, if you use kubectl-user to create a Pod, you will get the following prompt.

1

Error from server (Forbidden): error when creating "STDIN": pods "nginx" is forbidden: unable to validate against any pod security policy: []

Because sa fake user does not have the permission of psp example, you can check it by auth.

1

kubectl-user auth can-i use podsecuritypolicy/example

So the next step is to create the role, rolebinding.

4. Create a role with psp example use permission and bind to sa hellobaby

1
2
3
4
5
6
7
8

kubectl-admin create role psp:unprivileged --verb=use \
    --resource=podsecuritypolicy \
    --resource-name=example

kubectl-admin create rolebinding fake-user:psp:unprivileged \
    --role=psp:unprivileged \
    --serviceaccount=hellobaby:fake-user
rolebinding "fake-user:psp:unprivileged" created

Now go create the above Pod nginx again, and it will be created, and it will be created successfully, because this Pod does not request any special privileges and is a good boy.

What about adding the privileged fields and creating it again?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - image: nginx
    name: nginx
    securityContext:
      privileged: true

kubectl-user will return the following.

1

Error from server (Forbidden): error when creating "STDIN": pods "privileged" is forbidden: unable to validate against any pod security policy: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]

The tip is clear: Privileged containers are not allowed.

PodSecurityPolicy is in effect.

5. Pods not created directly by sa

Most of the time we don’t create Pods directly, but rather we create Deployment.

1
2
3
4
5
6
7
8
9

$ kubectl-user run nginx --image=nginx
deployment "nginx" created

$ kubectl-user get pods
No resources found.

$ kubectl-user get events | head -n 2
LASTSEEN   FIRSTSEEN   COUNT     NAME              KIND         SUBOBJECT                TYPE      REASON                  SOURCE                                  MESSAGE
1m         2m          15        nginx-7774d79b5   ReplicaSet                            Warning   FailedCreate            replicaset-controller                   Error creating: pods "nginx-7774d79b5-" is forbidden: no providers available to validate pod request

The message says that there are no providers to check for Pod requests, but we have already given sa hellobaby:fake-user authorization.

The reason is that this Pod is created by the replicaset controller, and the default kubectl run of the application specifies spec.template.spec.serviceAccount and spec.template.spec. serviceAccountName are both default, while the SA of our RBAC binding above is hellobaby:fake-user, not hellobaby:default.

How to solve it? The official website gives the method to bind the role psp:unprivileged to hellobaby:default again. It is true that this can solve the problem, but this solution does not make sense.

We know that a namespace can create more than one serviceaccount, if I need to grant different psp to different sa under the same namespace, then how should I give sa default authorization? Obviously it is impossible to bind.

In fact, I think this example from the official site doesn’t make sense and the kubectl run command is going to be eliminated.

A better approach would be to create a deployment and set its spec.template.spec.serviceAccount and spec.template.spec.serviceAccountName to the actual sa.

Here is an actual example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

metadata:
  labels:
    run: nginx-test
  name: nginx-test
  namespace: hellobaby
spec:
  replicas: 1
  selector:
    matchLabels:
      run: nginx-test
  template:
    metadata:
      labels:
        run: nginx-test
    spec:
      containers:
      - image: nginx
        imagePullPolicy: Always
        name: nginx-test
        #securityContext:
        #  privileged: true
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      serviceAccount: fake-user
      serviceAccountName: fake-user

At this point the replicaset controller will use sa hellobaby:fake-user to create a pod, and since the sa has been bound to the psp example, the Pod can be successfully created.

Ref:

• Pod Security Policies