Service is the core concept of the k8s networking part, and in k8s, Service is mainly responsible for four layers of load balancing. This article explains the network-related principles of k8s in terms of load balancing, extranet access, DNS service construction, and the Ingress seven-layer routing mechanism.

Service Explained

Service is the main mechanism used to implement the application to provide services to the outside world.

k8s service

As shown in the figure above, Service is a layer of abstraction for Pod, mainly through TCP/IP mechanism and listening IP and port number to provide services to the outside world. Unlike Pods, once a Service is created, the system distributes a ClusterIP for it (or you can specify it yourself), and it will not change during its lifetime.

Service creation

Command Line Quick Creation

After creating an RC, you can quickly create a corresponding Service by using the command line kubectl expose. For example, there is now an rc named hdls.

1

kubectl expose rc hdls

The ClusterIP of a Service created in this way is automatically assigned to it, and the port number of the Service is copied from the containerPort in the Pod.

created via YAML

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

apiVersion: v1
kind: Service
metadata:
  name: hdls
spec:
  ports:
  - port: 8080   # Service 的虚拟端口
    targetPort: 8000  # 指定后端 Pod 的端口号
  selector:  # Label 选择器
    app: hdls

Once the YAML file is defined, it can be created with the command kubectl create -f <service.yml>. The Service definition requires the following key fields to be specified.

  • ports
    • port: the virtual port of the Service.
    • targetPort: The port number of the backend Pod, if not filled in, it will be the same as the Service’s port by default.
  • selector: Label selector, specifying the Label owned by the backend Pod.

Load Distribution Policies

k8s provides two load distribution strategies.

  • RoundRobin: Polling method. That is, polling to forward requests to each Pod on the backend.
  • SessionAffinity: Session holding mode based on client IP address. That is, requests from clients with the same IP address are forwarded to the same Pod.

By default, k8s uses polling mode for routing. However, we can also enable SessionAffinity mode by setting “service.spec.SessionAffinity” to ClusterIP.

Some special cases

Headless Service.

In this case, k8s is implemented through the concept of Headless Service, where the Service is not given a ClusterIP (no ingress IP) and only the list of back-end Pods is returned to the calling client via Label Selector.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

apiVersion: v1
kind: Service
metadata:
  name: hdls
spec:
  ports:
  - port: 8080   
    targetPort: 8000 
  clusterIP: None
  selector: 
    app: hdls

The Service does not have a virtual ClusterIP and access to it can get a list of all Pods with app=hdls, the client needs to implement its own responsible balancing policy and then determine which Pod to access.

No LabelSelector Service.

Typically, applications need to connect to an external database as a backend service, or a service in another cluster or namespace as a backend service. In these cases, this can be achieved by creating a Service without Label Selector.

1
2
3
4
5
6
7
8

apiVersion: v1
kind: Service
metadata:
  name: hdls
spec:
  ports:
  - port: 8080   
    targetPort: 8000

The Service does not have a tag selector, i.e., it cannot select a backend Pod; in this case, the Endpoint is not created automatically, and an Endpoint with the same name as the Service needs to be created manually to point to the actual backend access address.

1
2
3
4
5
6
7
8
9

apiVersion: v1
kind: Endpoints
metadata:
  name: hdls  # 与 Service 同名
subsets:
  - addresss:
   - IP: 1.2.3.4   # 用户指定的 IP 
   ports:
   - port: 8000

In this case, the Endpoint is created as in the YAML above, and accessing the Service without Label Selector routes the request to the user-specified Endpoint.

Multi-Port Service..

Define multiple ports in service.spec.ports, including the name and protocol of the specified port.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

apiVersion: v1
kind: Service
metadata:
  name: hdls
spec:
  ports:
  - name: dns
    port: 8080   
    protocol: TCP
  - name: dns-tcp
    port: 8080 
    protocol: UDP
  selector: 
    app: hdls

Extranet access

Pods and Services are virtual concepts inside the k8s cluster, so they are not accessible to clients outside the cluster. However, under some special conditions, we need to have external access to Pod or Service, then we need to map the port number of Pod or Service to the host, so that clients can access the container application through the physical machine.

Extranet access to Pod

Map the port number of the container application to the physical machine. There are two ways to do this, as follows.

Set the container level hostPort

This kind is to map the port number of the container application to the physical machine. The settings are as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

apiVersion: v1
kind: Pod
metadata:
  name: hdls-pod
spec:
  containers:
  - name: hdls-container
   image: ***
   ports:
   - containerPort: 8000
     hostPort: 8000

Set Pod level hostNetwork=true

This maps all container port numbers in that Pod directly to the physical machine. Note that in the container ports definition section, if hostPort is not specified, the default hostPort=containerPort, and if hostPort is set, hostPort must be equal to containerPort. set as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

apiVersion: v1
kind: Pod
metadata:
  name: hdls-pod
spec:
  hostNetwork: true
  containers:
  - name: hdls-container
   image: ***
   ports:
   - containerPort: 8000

Extranet Access Service

There are also two ways to do this.

Set nodePort mapping to physical machine

First you need to set the nodePort mapping to the physical machine, and you need to set the Service type to NodePort.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

apiVersion: v1
kind: Service
metadata:
  name: hdls
spec:
  type: NodePort  # 指定类型为 NodePort
  ports:
  - port: 8080 
    targetPort: 8000 
    nodePort: 8000   # 指定 nodePort
  selector: 
    app: hdls

Set up LoadBalancer mapping to the LoadBalancer address provided by the cloud service provider

This usage is only used in scenarios where the service is set up on the cloud platform of the public cloud service provider. The service.status.loadBalancer.address.ip needs to be set to the IP of the load balancer provided by the cloud service provider. The load balancing implementation depends on the implementation mechanism of the LoadBalancer provided by the cloud provider.

DNS build

To enable mutual access to services within the cluster by service name, a virtual DNS service needs to be created to resolve the service name to the ClusterIP.

DNS provided by k8s

The DNS service provided by k8s is called skydns and consists of the following four components.

  • etcd: DNS storage.
  • kube2sky: registers the Service in k8s Master to etcd; * kube2sky: registers the Service in k8s Master to etcd.
  • skyDNS: DNS domain name resolution service.
  • healthz: health check for skyDNS.

The skyDNS service consists of an RC and a Service. In the configuration file of the RC, four containers, etcd / kube2sky / skydns / healthz, need to be defined to ensure that the DNS service works properly. It is important to note that:

  1. the kube2sky container needs to access the k8s Master, so you need to configure the IP address and port of the physical host where the Master is located in the configuration file for it.
  2. you need to set the startup parameter --domain of kube2sky and skydns containers to the domain name of the Service in the k8s cluster. After the container is started, kube2sky will monitor the definition of all services in the cluster through API Server, generate the corresponding records and save them to etcd; 3. The skydns startup parameter -addr=<IP:Port> indicates that the local TCP and UDP port provides services.

In the DNS Service configuration file, we need to specify the ClusterIP of skydns, which will be used by each Node kubelet and will not be automatically assigned by the system; in addition, this IP needs to be in the kube-apiserver startup parameter -service-cluster-ip-range.

Before the skydns container is created, you need to modify the kubelet startup parameters on each Node: * –cluster_dns

  • --cluster_dns=<dns_cluster_ip>, dns_cluster_ip is the ClusterIP of the DNS service.
  • --cluster_domain=<dns_domain>, dns_domain is the domain name set in the DNS service.

How DNS works

  1. first kube2sky container application gets all the service information in the cluster by calling k8s Master’s API, and continuously monitors the new service generation and writes it to etcd;
  2. According to the kubelet startup parameters, kubelet will set the DNS resolution configuration file /etc/resolv.conf to add a nameserver configuration and a search configuration in each newly created Pod, and the nameserver will actually access the DNS resolution service provided by skydns on the corresponding port. The DNS resolution service provided by skydns on the corresponding port.
  3. Finally, the application can access the service just by its name, just like a website domain name.

Ingress

Service works at TCP/IP layer, and Ingress forwards different URL access requests to different services at the backend, implementing the business routing mechanism at HTTP layer. In k8s, you need to combine Ingress and Ingress Controller to form a complete HTTP load balancing.

Ingress Controller

The Ingress Controller is used to provide a unified entry point for all back-end services, and needs to implement load distribution rules based on different HTTP URLs forwarded backwards.

  • Listen to APIServer and get all Ingress definitions.
  • Generate the required Nginx configuration file /etc/nginx/nginx.conf based on the Ingress definitions.
  • Execute nginx -s reload to reload the contents of the nginx.conf configuration file.

Defining Ingress

There is a separate resource in k8s called Ingress where you can set forwarding rules to the backend service in its configuration file. For example, define an ingress.yml for hdls.me.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

apiVersion: extensions/v1beta1
kind: Ingress
metadata: 
  name: hdls-ingress
spec:
  rules:
  - host: hdls.me
   http:
     paths:
     - path: /web
       backend:
         serviceName: hdls
         servicePort: 8000

Finally, use kubectl create -f ingress.yml to create Ingress, and log in to the nginx-ingress Pod to see the contents of its automatically generated nginx.conf configuration file.