Normally, when a client in the cluster connects to the service, the Pod supporting the service can get the IP address of the client, but when the connection is received through the node port, the source IP address of the packet will change because of the Source Network Address Translation (SNAT) performed on the packet, and the Pod on the backend cannot see the actual client IP, which is a problem for some applications, for example, the request log of nginx cannot get the exact IP of the client access.

For example, in our application below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  selector:
    app: nginx
  type: NodePort
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

You can see that the nginx service is automatically assigned a NodePort of 32761 after it is created directly.

1
2
3
4
5
6
7
8
9

$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP        28d
nginx        NodePort    10.106.190.194   <none>        80:32761/TCP   48m
$ kubectl get pods -o wide
NAME                              READY   STATUS    RESTARTS   AGE     IP             NODE         NOMINATED NODE   READINESS GATES
nginx-54f57cf6bf-nwtjp            1/1     Running   0          3m      10.244.3.15    ydzs-node3   <none>           <none>
nginx-54f57cf6bf-ptvgs            1/1     Running   0          2m59s   10.244.2.13    ydzs-node2   <none>           <none>
nginx-54f57cf6bf-xhs8g            1/1     Running   0          2m59s   10.244.1.16    ydzs-node1   <none>           <none>

We can see that the three Pods are assigned to three different nodes, and we can access our services through the NodePort of the master node. Since only the master node has access to the external network, we can see that the clientIP we get in the nginx pod log is 10.151.30.11, which is actually the master node’s internal IP, not the real IP address we expect from the browser.

1
2

$ kubectl logs -f nginx-54f57cf6bf-xhs8g
10.151.30.11 - - [07/Dec/2019:16:44:38 +0800] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" "-"

This is because we do not have a corresponding Pod on the master node, so when we access the application through the master node, we must need additional network hops to reach the Pod on other nodes, and during the hopping process, we see the IP of the master node because of the SNAT of the packets. externalTrafficPolicy to reduce the number of network hops.

1
2

spec:
  externalTrafficPolicy: Local

If externalTrafficPolicy=Local is configured in the Service and the external connection is opened through the service’s node port, the Service will proxy to the locally running Pod. For example, if we set the field to update, we can’t access the application through the NodePort of the master node because there is no Pod running on the master node, so we need to make sure the load balancer forwards the connection to a node with at least one Pod.

However, it should be noted that there is a drawback to using this parameter. Normally, requests are distributed evenly across all Pods, but with this configuration, the situation may not be the same. For example, if we have 3 Pods running on two nodes, and if Node A is running one Pod and Node B is running two Pods, if the load balancer distributes the connections evenly between the two nodes, the Pod on Node A will receive 50% of all requests, but the two Pods on Node B will only receive 25% each.

With the addition of externalTrafficPolicy: Local, the node receiving the request and the target Pod are both on the same node, so there is no additional network hopping (no SNAT is performed), so we can get the correct client IP, as shown below we fix the Pods to the master node.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

apiVersion: apps/v1
kind: Deployment
metadata:
  name:  nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      tolerations:
      - operator: "Exists"
      nodeSelector:
        kubernetes.io/hostname: ydzs-master
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
 externalTrafficPolicy: Local
  selector:
    app: nginx
  type: NodePort
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

After updating the service, and then accessing the service via NodePort, you can see that you are getting the correct client IP address.

1
2

$ kubectl logs -f nginx-ddc8f997b-ptb7b
182.149.166.11 - - [07/Dec/2019:17:03:43 +0800] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" "-"