Istio Sidecar injection mechanism

Service Mesh and Sidecar Concepts

Before understanding the injection mechanism of Sidecar, it’s important to clarify the what and why questions.

First, what is a Service Mesh?

Service Mesh, or translated as “Service Mesh”, is a configurable low-latency infrastructure layer designed to handle a large amount of network-based inter-process communication between application services through APIs (Application Programming Interfaces). The Service Grid ensures fast, reliable and secure communication between infrastructure services of containerized transient presence applications. Grids provide key functionality including service discovery, load balancing, encryption, observability, traceability, authentication and authorization, and support for circuit breaker mode. In fact, the purpose of Service Mesh is most simply stated as “takeover and governance of communication between applications”, where one of the most central points is that communication, Service Mesh-based service governance is to “tinker” with the communication or invocation process of services, away from this point, Service Mesh is meaningless, of course, the application seems The application also seems to be meaningless. The diagram below shows a typical Service Mesh infrastructure layer architecture.

As you can see, there are Control Plane and Data Plane, the main role of Control Plane is to control and distribute the governance rules, while the main role of Data Plane is to handle the communication process between service instances and implement the specified governance policies. In the Data Plane, you can see the protagonist of today, that is, Sidecar.

As indicated in the diagram, Sidecar is most accurately referred to as “Sidecar Proxy”, which is essentially a proxy component. This component is injected directly into the same Network Namesapce as the service instance. In Kubernetes, it is injected into the Pod. In this case, Sidecar shares the Pod Network Namespace with the service instance, and inbound and outbound traffic flowing through the instance can be handled with appropriate rules via iptables.

In this article, we analyze the injection mechanism of Sidecar based on Istio, a Service Mesh implementation.

Admission Controller and Admission Webhook

Sidecar’s injection relies on several Kubernetes concepts, the more central of which are the Admission Controller and Admission Webhook.

Admission Controller, as officially explained, is a gateway that intercepts API Server requests (authenticated) and can modify the request object or deny the request. In short, it can be thought of as an interceptor, similar to a middleware in a web framework, a means used by the Kubernetes API Server to intercept requests.

Why does Kubernetes introduce admission as a mechanism?

1. Although Kubernetes has Authentication & Authorization, which is an authentication and authentication mechanism, Authentication & Authorization runs in a filter and can only get the http request header and certificate, but not the request body. So you can’t do any operation on the requested object, because you can’t get the object.
2. The Admission Controller runs in the API Server’s add/delete handler and can naturally manipulate the API resource.

After the API Server receives the client request, it first authenticates and authenticates the request, and only after the authentication and authentication are passed will it proceed to the subsequent endpoint handler processing. kube-apiserver processes the resource request as follows.

As you can see, after Authentication & Authorization, the request is handed over to the Admission Controller for further processing, which involves two important phases of Admission, Mutating and Validating, the differences of which are as follows.

• Mutating: allows modifications to the request content.
• Validating: does not allow modification of the request content, but can determine whether to proceed with the request or reject it based on the content of the request.

Kubernetes provides a lot of built-in Admission Controller Plugin, and some common admission control policies can be found.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# The supported plugins are as follows
AlwaysAdmit, AlwaysDeny, AlwaysPullImages, 
DefaultStorageClass, DefaultTolerationSeconds, DenyEscalatingExec, 
DenyExecOnPrivileged, EventRateLimit, ExtendedResourceToleration, 
ImagePolicyWebhook, Initializers, LimitPodHardAntiAffinityTopology, 
LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, 
NamespaceExists, NamespaceLifecycle, NodeRestriction, 
OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, 
PersistentVolumeLabel, PodNodeSelector, PodPreset, PodSecurityPolicy, 
PodTolerationRestriction, Priority, ResourceQuota, SecurityContextDeny, 
ServiceAccount, StorageObjectInUseProtection, ValidatingAdmissionWebhook. 

Kubernetes provides so many Admission plugins, but they are not guaranteed to meet the needs of all developers. Therefore, Kubernetes also allows users to customize their own Admission Controller, and Kubernetes provides the Admission Webhook extension mechanism.

• MutatingAdmissionWebhook: modifying objects before they persist
• ValidatingAdmissionWebhook: makes changes before the object is persisted

Admission Webhook is a synchronous call that requires the user to develop and deploy their own webhook server and create a custom configuration resource object: ValidatingWebhookConfiguration or MutatingWebhookConfiguration.

It can be said that with the Admission Webhook extension mechanism, it really paves the way for the Sidecar injection implementation later.

Istio Sidecar Auto-Injection Implementation

Sidecar Injector is the component in Istio that implements automatic Sidecar injection, which is running as the Kubernetes Admission Controller. Recall from the previous concept that the basic principle of the Admission Controller is to intercept requests from the Kube-apiserver, before object persistence and after authentication and authentication. As mentioned before, there are two types of Admission Controllers: one is built-in and the other is user-defined. The latter is where Kubernetes allows users to customize the admission controller in the form of a Webhook, and Sidecar Injector is one such special MutatingAdmissionWebhook.

The Sidecar injection process is shown in the following diagram.

As shown in the figure, Sidecar Injector only performs Sidecar container injection when creating Pods, and after the Pod creation request arrives at Kube-apiserver, it first performs authentication and authentication, and then in the access control phase, Kube-apiserver calls Sidecar Injector in a REST manner simultaneously. Webhook service to init and inject the istio-proxy container, and finally store the Pod object persistently in the etcd.

See also the configuration of MutatingWebhookConfiguration.

1

kubectl get MutatingWebhookConfiguration istio-sidecar-injector -n istio-system -o yaml

istio-sidecar-injector MutatingWebhookConfiguration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
            {"apiVersion":"admissionregistration.k8s.io/v1beta1","kind":"MutatingWebhookConfiguration","metadata":{"annotations":{},"labels":{"app":"sidecar-injector","install.operator.istio.io/owning-resource":"installed-state","istio.io/rev":"default","operator.istio.io/component":"Pilot","operator.istio.io/managed":"Reconcile","operator.istio.io/version":"1.6.2","release":"istio"},"name":"istio-sidecar-injector"},"webhooks":[{"clientConfig":{"caBundle":"","service":{"name":"istiod","namespace":"istio-system","path":"/inject"}},"failurePolicy":"Fail","name":"sidecar-injector.istio.io","namespaceSelector":{"matchLabels":{"istio-injection":"enabled"}},"rules":[{"apiGroups":[""],"apiVersions":["v1"],"operations":["CREATE"],"resources":["pods"]}],"sideEffects":"None"}]}
  creationTimestamp: "2020-06-16T06:47:00Z"
  generation: 2
  labels:
    app: sidecar-injector
    install.operator.istio.io/owning-resource: installed-state
    istio.io/rev: default
    operator.istio.io/component: Pilot
    operator.istio.io/managed: Reconcile
    operator.istio.io/version: 1.6.2
    release: istio
  managedFields:
		...
webhooks:
- admissionReviewVersions:
  - v1beta1
  clientConfig:
    caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ...
    service:
      name: istiod
      namespace: istio-system
      path: /inject
      port: 443
  failurePolicy: Fail
  matchPolicy: Exact
  name: sidecar-injector.istio.io
  namespaceSelector:
    matchLabels:
      istio-injection: enabled
  objectSelector: {}
  reinvocationPolicy: Never
  rules:
  - apiGroups:
    - ""
    apiVersions:
    - v1
    operations:
    - CREATE
    resources:
    - pods
    scope: '*'
  sideEffects: None
  timeoutSeconds: 30

By default, Sidecar Injector only takes effect for the creation of Pod resource objects under the namespace whose tag matches istio-injection: enabled. The access path to the Webhook service is /inject, and the address and access credentials are configured under the clientConfig field. The Istio Sidecar Injector component is implemented by the sidecar-injector process, and the Sidecar Injector implementation consists of two main parts.

• MutatingWebhookConfiguration
• Webhook Server, which automatically injects Sidecar containers for application workloads

Sidecar Injector handles the AdmissionRequest requests from Kube-apiserver as a lightweight HTTPS server.

Typically the Pod Sidecar container is injected by the following steps:

1. parsing the Webhook REST request to deserialize the raw AdmissionReview data.
2. parsing the Pod and deserializing the AdmissionRequest in the AdmissionReview.
3. render the Sidecar configuration template using the Pod and the grid configuration.
4. create a JSON patch using the Pod and the rendered template.
5. construct the AdmissionResponse.
6. construct the AdmissionReview and send it to the HTTP client, the Kube-apiserver, after JSON encoding.

You can view the injected configuration item istio-sidecar-injector.

1

kubectl describe configmap istio-sidecar-injector -n istio-system

istio-sidecar-injector configmap(Abbreviated version)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

Name:         istio-sidecar-injector
Namespace:    istio-system
Labels:       install.operator.istio.io/owning-resource=installed-state
              istio.io/rev=default
              operator.istio.io/component=Pilot
              operator.istio.io/managed=Reconcile
              operator.istio.io/version=1.6.2
              release=istio
Annotations:
Data
====
config:
----
policy: enabled
...
template: |
  rewriteAppHTTPProbe: {{ valueOrDefault .Values.sidecarInjectorWebhook.rewriteAppHTTPProbe false }}
  initContainers:
  	...
  - name: istio-validation
  	...
  - name: istio-init  
		...
    args:
    - istio-iptables
    - "-p"
    - 15001
		...
  containers:
  - name: istio-proxy
    ports:
    - containerPort: 15090
      protocol: TCP
      name: http-envoy-prom
    args:
		...
    env:
    - name: JWT_POLICY
      value: {{ .Values.global.jwtPolicy }}
    ...

ConfigMap stores the default injection policies and sidecar injection templates. policy:

• disabled: sidecar injectors are not injected into the pod by default. Adding the annotation sidecar.istio.io/inject to the pod template definition with a value of true will enable the injection feature.
• enabled: sidecar injector will be injected into the pod by default. Adding the annotation sidecar.istio.io/inject to the pod template definition with a value of false will disable the injection.