部署kubernetes-dashboard并配置ServiceAccount和登录鉴权

• "种草" kubernetes-dashboard
• 安装部署dashboard
• 创建用于登录面板的ServiceAccount
• 权限控制

"种草" kubernetes-dashboard

Kubernetes Dashboard 是通用的用于管理 Kubernetes 集群的 WebUI面板

kubernetes-dashboard 代码库 readme 中对自己的介绍:

Kubernetes Dashboard is a general purpose, web-based UI for Kubernetes clusters. It allows users to manage applications running in the cluster and troubleshoot them, as well as manage the cluster itself.

通过这个面板, 可以让我们非常直观看到pod的状态, 创建时间, 标签, 运行在哪个结点等元数据

还能看到容器中定义的环境变量, 挂载的磁盘, liveness和readiness探针等信息

有了这个面板, 不光是pod, 可以对几乎 kubernetes 中的任何资源(前提是当前用户有足够的权限) 查看和管理

再介绍一个这个面板我自己最常用的功能: 重启pod

确实通过命令kubectl rollout restart -n prod deployment mysql-8-0可以实现同样的操作, 不过有了这个面板, 鼠标点点就能完成岂不美哉, 重启前还会弹出确认框提示你是不是确定要重启, 还贴心的把用于重启命令给我们展示出来了, awesome! 再也不怕忘记命令每次都要去google一下文档啦!

---

这个面板可以使用kubernetes的ServiceAccount来登录鉴权, 对权限控制的粒度可以做到非常细致, 别急, 往下看!

安装部署dashboard

官方的deployment.yaml

这份文件中, 首先定义了个Namespace -> kubernetes-dashboard, 然后的所有资源都在这个Namespace下;

先看2个Deployment的定义, 分别是kubernetes-dashboard和dashboard-metrics-scraper

点击展开 -> Deployment: kubernetes-dashboard

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.7.0
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --auto-generate-certificates
            - --namespace=kubernetes-dashboard
            # 通过nginx-ingress暴露出去, 这里不需要ssl了, 没有这个标志位会导致无法正常登录
            - --enable-insecure-login=true
          volumeMounts:
            - name: kubernetes-dashboard-certs
              mountPath: /certs
              # Create on-disk volume to store exec logs
            - mountPath: /tmp
              name: tmp-volume
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            initialDelaySeconds: 30
            timeoutSeconds: 30
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      volumes:
        - name: kubernetes-dashboard-certs
          secret:
            secretName: kubernetes-dashboard-certs
        - name: tmp-volume
          emptyDir: {}
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule

点击展开 -> Deployment: dashboard-metrics-scraper

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: dashboard-metrics-scraper
  template:
    metadata:
      labels:
        k8s-app: dashboard-metrics-scraper
    spec:
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: dashboard-metrics-scraper
          image: kubernetesui/metrics-scraper:v1.0.8
          ports:
            - containerPort: 8000
              protocol: TCP
          livenessProbe:
            httpGet:
              scheme: HTTP
              path: /
              port: 8000
            initialDelaySeconds: 30
            timeoutSeconds: 30
          volumeMounts:
          - mountPath: /tmp
            name: tmp-volume
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      volumes:
        - name: tmp-volume
          emptyDir: {}

再看ServiceAccount: kubernetes-dashboard; 上面两个Deployment kubernetes-dashboard和dashboard-metrics-scraper 中的容器都绑定到了这个ServiceAccount

点击展开 ServiceAccount: kubernetes-dashboard

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard

deployment.yaml中还定义了ClusterRole: kubernetes-dashboard和Role: kubernetes-dashboard, ClusterRole Role中定义了面板所需的权限

ClusterRole和Role的定义

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
rules:
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
    verbs: ["get", "update", "delete"]
    # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["kubernetes-dashboard-settings"]
    verbs: ["get", "update"]
    # Allow Dashboard to get metrics.
  - apiGroups: [""]
    resources: ["services"]
    resourceNames: ["heapster", "dashboard-metrics-scraper"]
    verbs: ["proxy"]
  - apiGroups: [""]
    resources: ["services/proxy"]
    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
    verbs: ["get"]

---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
rules:
  # Allow Metrics Scraper to get metrics from the Metrics server
  - apiGroups: ["metrics.k8s.io"]
    resources: ["pods", "nodes"]
    verbs: ["get", "list", "watch"]

ClusterRole, Role与 前面提到的 ServiceAccount:kubernetes-dashboard 通过 ClusterRoleBinding 和 RoleBinding 绑定到了一起, 就是给 ServiceAccount:kubernetes-dashboard 授权, 让它可以访问集群里的资源

ClusterRoleBinding, RoleBinding 绑定 ServiceAccount 与 ClusterRole, Role

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

最后就是 Service, ConfigMap, 和几个ssl相关的 Secret, 几个 Secret 的定义对我来说是不重要的, 因为上面Deployment容器的启动命令里, 我加上了 --enable-insecure-login=true, 我在外部通过nginx-ingress暴露服务, 配置好了ingress的部分配置ssl, 所里这里就无所谓了

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 8000
      targetPort: 8000
  selector:
    k8s-app: dashboard-metrics-scraper

---

kind: ConfigMap
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-settings
  namespace: kubernetes-dashboard

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-csrf
  namespace: kubernetes-dashboard
type: Opaque
data:
  csrf: ""

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-key-holder
  namespace: kubernetes-dashboard
type: Opaque

配置Ingress, 向外暴露服务

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: kubernetes-dashboard
  name: kubernetes-dashboard-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: http
spec:
  ingressClassName:
  tls:
    - hosts:
        - <这里填域名>
      secretName: <这里填集群内配置好的Secret证书>
  rules:
    - host: <这里填域名>
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: kubernetes-dashboard
              port:
                number: 80

创建用于登录面板的ServiceAccount

新建account.yaml中定义ServiceAccount并绑定到内置的ClusterRole

定义两个 ServiceAccount 分别是 root 和 readonly

• root 绑定到内置的 ClusterRole cluster-admin
• readonly 绑定到内置的 ClusterRole view
• kubectl get clusterrole 可以列表当前集群中已存在的角色

apiVersion: v1
kind: ServiceAccount
metadata:
  name: root
  namespace: kubernetes-dashboard

---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: readonly
  namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: root
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: root
  namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: readonly
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
- kind: ServiceAccount
  name: readonly
  namespace: kubernetes-dashboard

kubectl apply -f account.yaml 应用后, root 和 readonly 就创建好了, 然后我们去生成 Token 输入到面板的登录界面中就可以登录了

为 root 生成有效期30天 Token 的命令 kubectl create token --namespace kubernetes-dashboard --duration 2592000s root

用 Token 登录

权限控制

root 账户绑定的角色是内置的 cluster-admin, 可以对集群内的一切资源进行查看或修改, 删除等操作

readonly 账户绑定的角色是内置的 view, 拥有对集群内大部分资源的只读权限

也可以自己新建一个角色, 授予这个新角色某些指定权限, 例如下面的示例, 创建了一个名为 test 的 ClusterRole, 对pods有只读权限, 然后将这个角色与 ServiceAccount 绑定

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: test
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "watch", "list"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: root-to-test
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: test
subjects:
- kind: ServiceAccount
  name: root
  namespace: kubernetes-dashboard