kubernetes之DaemonSet以及滚动更新

1.什么是DaemonSet？

1.1DaemonSet是Pod控制器的又一种实现方式，用于在集群中的全部节点上同时运行一份指定的Pod资源副本，后续加入集群的节点也会自动创建一个相关的Pod对象，当从集群移除节点时，此类Pod对象也将被自动回收无需创建。管理员也可以使用"节点选择器"以及标签指定仅在部分具有特定特征的节点上运行指定的Pod对象。

1.2官方文档: https://kubernetes.io/zh-cn/docs/concepts/workloads/controllers/daemonset/

2.DaemonSet的用法

• 在每个节点上运行集群存储守护进程，如glusterd或Ceph；
• 在每个节点上运行日志收集守护进程，如fluentd、logstash；
• 在每个节点上运行监控系统代理程序，如Prometheus Node Exporter；
• 在每个节点上运行网络插件为Pod提供网络服务，如flannel、Calico；
• 在每个节点上运行kube-Proxy通过API-Server持续监控各个Service及其关联的Pod对象，并将其创建或变动实时反应至当前节点相应的iptables、ipvs规则上；

3.DaemonSet资源规范

DaemonSet是标准的K8s资源类型，它在Spec字段中嵌套了Selector、Template和minReadySeconds，并且功能和用法基本相同，但它不支持使用Replicas，DaemonSet并不是基于期望的副本数来控制Pod资源数量，而是基于节点数量来控制Pod数量；

apiVersion: apps/v1   # API群组及版本；
kind: DaemonSet      # 资源类型；
metadata:             # Pod元数据；
   name:              # 资源名称，在作用域中要唯一；
   namespace: <string>         # 名称空间，DaemonSet隶属名称空间级别；
spec:
  minReadySeconds: <integer>   # Pod就绪后多少秒内任一容器无Crash方可为就绪，来控制滚动更新的速度，默认值为0，表示新建的Pod对象一旦"就绪"将立即被视作可用，随后开始下一轮更新。如果设定了spec.minReadySeconds: 3表示新建的Pod对象至少要成功运行多久才会被视作可用，即就绪之后还要等待指定的3s才能开始下一批次的更新，在一个批次内新建的所有Pod就绪探测失败，都会导致滚动更新被终止，因此为minReadySeconds设定一个合理的值，不仅能够减缓更新的速度，还能够让Deployment提前发现一部分程序因为Bug导致的升级故障，生产环境中为Pod设置指针探测。
  selector: <Object>           # 标签选择器，必须匹配template字段中Pod模板中的标签；
  template: <Object>           # Pod模板对象；
    metadata: <Object>         # Pod名称
    spec: <Object>             # Pod详情
  revisionHistoryLimit: <integer>  # 滚动更新历史记录数量，默认为10；
  updateStrategy: <Object>               # 滚动更新策略；
    type: <string>                 # 滚动更新类型，可用值有OnDelete和Rollingupdate；
    rollingUpdate: <Object>       # 滚动更新参数，专用于RollingUpdate类型；
      maxSurge: <string>           # 更新期间可比期望的Pod数量多出的数量和比例；
      maxUnavailable: <string>     # 更新期间可比期望的Pod数量缺少的数量或比例；

4.DaemonSet部署Node_exporter示例

• Github地址:https://github.com/prometheus/node_exporter
• DockerHub地址:https://hub.docker.com/r/prom/node-exporter
• Kubernetes1.8版本起，DaemonSet也必须使用selector来匹配Pod模板中指定的标签，而且它也支持matchLabels和matchExpressions两种标签选择器；

# 示例配置文件
root@kubernetes-master01:~# cat daemonSet-node_exporter.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: default
spec:
  selector:
    matchLabels:
      app: node-exporter

  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      hostNetwork: true                    # 共享主机网络
      hostPID: true                        # 获取主机的PID
      containers:
      - name: prometheus-node-exporter
        image: prom/node-exporter:v0.18.0
        ports:
        - name: node-ex-http
          containerPort: 9100
          hostPort: 9100                  # 监听在节点的9100端口上面，节点的9100，实际就在访问这个Pod
        livenessProbe:
          tcpSocket:
            port: node-ex-http
          initialDelaySeconds: 5

        readinessProbe:
          httpGet:
            path: '/metrics'
            port: node-ex-http
          initialDelaySeconds: 5

4.1 查看Pod实例；

root@kubernetes-master01:~# kubectl get pods -o wide -l app=node-exporter
NAME                  READY   STATUS    RESTARTS   AGE   IP           NODE                NOMINATED NODE   READINESS GATES
node-exporter-l7lfh   1/1     Running   0          55s   xx.0.0.xx   kubernetes-xxx  <none>           <none>
node-exporter-qljd6   1/1     Running   0          55s   xx.0.0.xx   kubernetes-xxx  <none>           <none>

4.2 describe命令查看详细信息；

# DaemonSet也可以简写为ds；
root@kubernetes-master01:~# kubectl get daemonset
NAME            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
node-exporter   2         2         2       2            2           <none>          3m9s

#
root@kubernetes-master01:~# kubectl describe ds node-exporter
Name:           node-exporter
Selector:       app=node-exporter
Node-Selector:  <none>
Labels:         <none>
Annotations:    deprecated.daemonset.template.generation: 1
Desired Number of Nodes Scheduled: 2   # 期望的Pod副本数
Current Number of Nodes Scheduled: 2   #
Number of Nodes Scheduled with Up-to-date Pods: 2
Number of Nodes Scheduled with Available Pods: 2
Number of Nodes Misscheduled: 0
Pods Status:  2 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:   # 两个Pod都是Running
  Labels:  app=node-exporter
  Containers:
   prometheus-node-exporter:
    Image:        prom/node-exporter:v0.18.0  # Pod使用的镜像
    Port:         9100/TCP                     # Pod的端口
    Host Port:    9100/TCP                     #  开启了共享宿主机端口
    Liveness:     tcp-socket :node-ex-http delay=5s timeout=1s period=10s #success=1 #failure=3        # 探针检测
    Readiness:    http-get http://:node-ex-http/metrics delay=5s timeout=1s period=10s #success=1 #failure=3   # 探针检测
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:     # 调度给两个节点
  Type    Reason            Age    From                  Message
  ----    ------            ----   ----                  -------
  Normal  SuccessfulCreate  4m26s  daemonset-controller  Created pod: node-exporter-qljd6
  Normal  SuccessfulCreate  4m26s  daemonset-controller  Created pod: node-exporter-l7lfh

4.3 node_exporter默认监听在TCP的9100端口，可以向任意一节点发起请求也是没有问题；

root@kubernetes-master01:~# curl -s xx.0.0.xx:9100/metrics | grep memory
# HELP node_memory_Active_anon_bytes Memory information field Active_anon_bytes.
# TYPE node_memory_Active_anon_bytes gauge
node_memory_Active_anon_bytes 3.01551616e+08
# HELP node_memory_Active_bytes Memory information field Active_bytes.
# TYPE node_memory_Active_bytes gauge
node_memory_Active_bytes 8.14239744e+08
# HELP node_memory_Active_file_bytes Memory information field Active_file_bytes.
# TYPE node_memory_Active_file_bytes gauge
node_memory_Active_file_bytes 5.12688128e+08
# HELP node_memory_AnonHugePages_bytes Memory information field AnonHugePages_bytes.
# TYPE node_memory_AnonHugePages_bytes gauge
node_memory_AnonHugePages_bytes 0

5.DaemonSet更新方式

1.DaemonSet自1.16版本开始也支持滚动更新机制，相关配置定义在spec.update-Strategy嵌套字段中。目前它支持RollingUpdate(滚动更新)和OnDelete(删除式更新)两种更新策略。滚动更新为默认的更新策略。工作逻辑类似于Deployment控制，不过仅支持使用maxUnavailable属性定义最大不可用Pod资源的副本数(默认为1)而删除式更新的方式则是在删除相应节点的Pod资源后重建并更新为新版本；

5.1 RollingUpdate更新示例:

# image: prom/node-exporter:v0.18.0修改为0.18.1
root@kubernetes-master01:~# cat daemonset-node_exporter.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: default
spec:
  minReadySeconds: 3
  revisionHistoryLimit: 20
  updateStrategy:
    type: RollingUpdate  # 更新策略
    rollingUpdate:
      maxUnavailable: 1
  selector:
    matchLabels:
      app: node-exporter

  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      hostNetwork: true                    # 共享主机网络
      hostPID: true                        # 获取主机的PID
      containers:
      - name: prometheus-node-exporter
        image: prom/node-exporter:v0.18.1
        ports:
        - name: node-ex-http
          containerPort: 9100
          hostPort: 9100                  # 监听在节点的9100端口上面，节点的9100，实际就在访问这个Pod
        livenessProbe:
          tcpSocket:
            port: node-ex-http
          initialDelaySeconds: 5

        readinessProbe:
          httpGet:
            path: '/metrics'
            port: node-ex-http
          initialDelaySeconds: 5

5.1.1执行滚动更新及状态；

root@kubernetes-master01:~# kubectl apply -f daemonset-node_exporter.yaml  && kubectl rollout status daemonset node-exporter
daemonset.apps/node-exporter configured
Waiting for daemon set "node-exporter" rollout to finish: 0 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 0 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 0 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 1 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 1 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 1 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 1 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 1 of 2 updated pods are available...
Waiting for daemon set "node-exporter" rollout to finish: 1 of 2 updated pods are available...
daemon set "node-exporter" successfully rolled out

5.1.2默认的RollingUpdate策略将采用一次更新一个Pod对象，待新建的Pod对象就绪后，再更新下一个Pod对象，直到全部完成。可以看到镜像版本为v0.18.1

root@kubernetes-master01:~# kubectl describe daemonsets.apps node-exporter
Pod Template:
  Labels:  app=node-exporter
  Containers:
   prometheus-node-exporter:
    Image:        prom/node-exporter:v0.18.1  # 镜像版本已经为0.18.1
Events:
  Type    Reason            Age    From                  Message
  ----    ------            ----   ----                  -------
  Normal  SuccessfulCreate  46m    daemonset-controller  Created pod: node-exporter-qljd6
  Normal  SuccessfulCreate  46m    daemonset-controller  Created pod: node-exporter-l7lfh
  Normal  SuccessfulDelete  2m34s  daemonset-controller  Deleted pod: node-exporter-qljd6
  Normal  SuccessfulCreate  2m32s  daemonset-controller  Created pod: node-exporter-q479h
  Normal  SuccessfulDelete  2m21s  daemonset-controller  Deleted pod: node-exporter-l7lfh
  Normal  SuccessfulCreate  2m11s  daemonset-controller  Created pod: node-exporter-8c2mr

5.2 OnDelete更新示例:

# 修改镜像版本为latest
root@kubernetes-master01:~# cat daemonset-node_exporter.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: default
spec:
  minReadySeconds: 3
  revisionHistoryLimit: 10
  updateStrategy:      # 滚动更新策略
    type: OnDelete     #  使用OnDelete
    rollingUpdate:
      maxUnavailable: 1
  selector:
    matchLabels:
      app: node-exporter

  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      hostNetwork: true                    # 共享主机网络
      hostPID: true                        # 获取主机的PID
      containers:
      - name: prometheus-node-exporter
        image: prom/node-exporter:latest  # 修改镜像版本为latest。
        ports:
        - name: node-ex-http
          containerPort: 9100
          hostPort: 9100                  # 监听在节点的9100端口上面，节点的9100，实际就在访问这个Pod
        livenessProbe:
          tcpSocket:
            port: node-ex-http
          initialDelaySeconds: 5

        readinessProbe:
          httpGet:
            path: '/metrics'
            port: node-ex-http
          initialDelaySeconds: 5

5.2.1 由于OnDelete并非自动完成升级，它需要管理员手动去删除Pod，然后重新拉起新的Pod，才能完成更新。(对于升级有着先后顺序的软件，这种方法非常有用；)

5.2.2删除Pod；

root@kubernetes-master01:~# kubectl delete pods node-exporter-8c2mr
pod "node-exporter-8c2mr" deleted
root@kubernetes-master01:~# kubectl delete pods node-exporter-q479h
pod "node-exporter-q479h" deleted

5.2.3查看Pod已经被拉起；

root@kubernetes-master01:~# kubectl get pods -l app=node-exporter
NAME                  READY   STATUS    RESTARTS   AGE
node-exporter-fbmkv   1/1     Running   0          99s
node-exporter-hv69b   1/1     Running   0          85s

5.2.4查看Events和版本；

root@kubernetes-master01:~# kubectl describe ds node-exporter
Name:           node-exporter
Selector:       app=node-exporter
Node-Selector:  <none>
Labels:         <none>
Annotations:    deprecated.daemonset.template.generation: 3
Desired Number of Nodes Scheduled: 2
Current Number of Nodes Scheduled: 2
Number of Nodes Scheduled with Up-to-date Pods: 2
Number of Nodes Scheduled with Available Pods: 2
Number of Nodes Misscheduled: 0
Pods Status:  2 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=node-exporter
  Containers:
   prometheus-node-exporter:
    Image:        prom/node-exporter:latest
    Port:         9100/TCP
    Host Port:    9100/TCP
    Liveness:     tcp-socket :node-ex-http delay=5s timeout=1s period=10s #success=1 #failure=3
    Readiness:    http-get http://:node-ex-http/metrics delay=5s timeout=1s period=10s #success=1 #failure=3
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:
  Type    Reason            Age    From                  Message
  ----    ------            ----   ----                  -------
  Normal  SuccessfulCreate  3m31s  daemonset-controller  Created pod: node-exporter-fbmkv
  Normal  SuccessfulCreate  3m17s  daemonset-controller  Created pod: node-exporter-hv69b

root@kubernetes-master01:~# kubectl get pods node-exporter-fbmkv  -o yaml
- containerID: docker://03e895265b19f82b3086c3bd7c013f7fa5edea96551a669aac164b92ea2e26b5
    image: prom/node-exporter:latest