部署k8s的heapster监控

Heapster是容器集群监控和性能分析工具，天然的支持Kubernetes和CoreOS

heapster监控目前官网已经不更新，部署学习使用

heapster: 收集监控数据

influxdb：数据库，存储数据

grafana：web页面展示

1、heapster安装包下载

地址：https://github.com/kubernetes-retired/heapster/releases

把对应的tar包下载

解压包，在路径：heapster-1.5.4\heapster-1.5.4\deploy\kube-config\rbac下找到heapster-rbac.yaml
在路径heapster-1.5.4\heapster-1.5.4\deploy\kube-config\influxdb下找到grafana.yaml，heapster.yaml，influxdb.yaml

2、部署influxdb

新版本k8sapi变动，修改Deployment  apiVersion为apiVersion: apps/v1

镜像修改为国内镜像源：image: registry.aliyuncs.com/google_containers/heapster-influxdb-amd64:v1.5.2

增加selector选择器

[root@k8s-master1 test2]# cat influxdb.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: monitoring-influxdb
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      task: monitoring
      k8s-app: influxdb
  template:
    metadata:
      labels:
        task: monitoring
        k8s-app: influxdb
    spec:
      containers:
      - name: influxdb
        image: registry.aliyuncs.com/google_containers/heapster-influxdb-amd64:v1.5.2
        volumeMounts:
        - mountPath: /data
          name: influxdb-storage
      volumes:
      - name: influxdb-storage
        emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  labels:
    task: monitoring
    # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)
    # If you are NOT using this as an addon, you should comment out this line.
    kubernetes.io/cluster-service: 'true'
    kubernetes.io/name: monitoring-influxdb
  name: monitoring-influxdb
  namespace: kube-system
spec:
  ports:
  - port: 8086
    targetPort: 8086
  selector:
    k8s-app: influxdb

 部署influxdb：
 # kubectl apply -f influxdb.yaml

3、部署heapster

新版本k8sapi变动，修改Deployment  apiVersion为apiVersion: apps/v1

镜像修改为国内镜像源：image: registry.aliyuncs.com/google_containers/heapster-amd64:v1.5.4

增加selector选择器

source参数修改为：- --source=kubernetes:https://kubernetes.default?kubeletHttps=true&kubeletPort=10250&insecure=true

不修改会提示报错，kubectl  logs可以查询到对应报错信息

[root@k8s-master1 test2]# cat heapster.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: heapster
  namespace: kube-system
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: heapster
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      task: monitoring
      k8s-app: heapster
  template:
    metadata:
      labels:
        task: monitoring
        k8s-app: heapster
    spec:
      serviceAccountName: heapster
      containers:
      - name: heapster
        image: registry.aliyuncs.com/google_containers/heapster-amd64:v1.5.4
        imagePullPolicy: IfNotPresent
        command:
        - /heapster
        - --source=kubernetes:https://kubernetes.default?kubeletHttps=true&kubeletPort=10250&insecure=true
        - --sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086
---
apiVersion: v1
kind: Service
metadata:
  labels:
    task: monitoring
    # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)
    # If you are NOT using this as an addon, you should comment out this line.
    kubernetes.io/cluster-service: 'true'
    kubernetes.io/name: Heapster
  name: heapster
  namespace: kube-system
spec:
  ports:
  - port: 80
    targetPort: 8082
  selector:
    k8s-app: heapster

用户权限，默认的没有create权限
# kubectl apply -f heapster-rbac.yaml

重新导出yaml文件，修改rule角色权限

# kubectl get ClusterRole system:heapster -o yaml > heapster_modify.yaml

# kubectl apply -f  heapster_modify.yaml

部署heapster
#kubectl apply -f heapster.yaml

查询角色权限，verbs中有了create权限
[root@k8s-master1 test2]# kubectl describe ClusterRole system:heapster
Name:         system:heapster
Labels:       kubernetes.io/bootstrapping=rbac-defaults
Annotations:  rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
  Resources               Non-Resource URLs  Resource Names  Verbs
  ---------               -----------------  --------------  -----
  events                  []                 []              [create get list watch]
  namespaces              []                 []              [create get list watch]
  nodes/stats             []                 []              [create get list watch]
  nodes                   []                 []              [create get list watch]
  pods                    []                 []              [create get list watch]
  deployments.extensions  []                 []              [get list watch]

4、部署grafana

新版本k8sapi变动，修改Deployment  apiVersion为apiVersion: apps/v1

镜像修改为国内镜像源：image: registry.aliyuncs.com/google_containers/heapster-grafana-amd64:v5.0.4

增加selector选择器

[root@k8s-master1 test2]# cat grafana.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: monitoring-grafana
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      task: monitoring
      k8s-app: grafana
  template:
    metadata:
      labels:
        task: monitoring
        k8s-app: grafana
    spec:
      containers:
      - name: grafana
        image: registry.aliyuncs.com/google_containers/heapster-grafana-amd64:v5.0.4
        ports:
        - containerPort: 3000
          protocol: TCP
        volumeMounts:
        - mountPath: /etc/ssl/certs
          name: ca-certificates
          readOnly: true
        - mountPath: /var
          name: grafana-storage
        env:
        - name: INFLUXDB_HOST
          value: monitoring-influxdb
        - name: GF_SERVER_HTTP_PORT
          value: "3000"
          # The following env variables are required to make Grafana accessible via
          # the kubernetes api-server proxy. On production clusters, we recommend
          # removing these env variables, setup auth for grafana, and expose the grafana
          # service using a LoadBalancer or a public IP.
        - name: GF_AUTH_BASIC_ENABLED
          value: "false"
        - name: GF_AUTH_ANONYMOUS_ENABLED
          value: "true"
        - name: GF_AUTH_ANONYMOUS_ORG_ROLE
          value: Admin
        - name: GF_SERVER_ROOT_URL
          # If you're only using the API Server proxy, set this value instead:
          # value: /api/v1/namespaces/kube-system/services/monitoring-grafana/proxy
          value: /
      volumes:
      - name: ca-certificates
        hostPath:
          path: /etc/ssl/certs
      - name: grafana-storage
        emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  labels:
    # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)
    # If you are NOT using this as an addon, you should comment out this line.
    kubernetes.io/cluster-service: 'true'
    kubernetes.io/name: monitoring-grafana
  name: monitoring-grafana
  namespace: kube-system
spec:
  # In a production setup, we recommend accessing Grafana through an external Loadbalancer
  # or through a public IP.
  # type: LoadBalancer
  # You could also use NodePort to expose the service at a randomly-generated port
  # type: NodePort
  ports:
  - port: 80
    targetPort: 3000
  selector:
    k8s-app: grafana
  type: NodePort

部署grafana
# kubectl apply -f grafana.yaml

5、查询部署资源

[root@k8s-master1 test2]# kubectl get all -n kube-system | egrep 'heapster|monitor'
pod/heapster-7f6787db47-xjtg2             1/1     Running   2          17h
pod/monitoring-grafana-745bf97858-5484w   1/1     Running   2          18h
pod/monitoring-influxdb-77864d8b5-dlwwz   1/1     Running   2          18h
service/heapster              ClusterIP   10.103.130.255   <none>        80/TCP                   17h
service/monitoring-grafana    NodePort    10.102.137.71    <none>        80:31526/TCP             18h
service/monitoring-influxdb   ClusterIP   10.102.238.82    <none>        8086/TCP                 18h
deployment.apps/heapster              1/1     1            1           17h
deployment.apps/monitoring-grafana    1/1     1            1           18h
deployment.apps/monitoring-influxdb   1/1     1            1           18h
replicaset.apps/heapster-7f6787db47             1         1         1       17h
replicaset.apps/monitoring-grafana-745bf97858   1         1         1       18h
replicaset.apps/monitoring-influxdb-77864d8b5   1         1         1       18h

6、可以使用top命令查询node，pod等资源监控数据。这个需要等一段时间才会有数据

[root@k8s-master1 test2]# kubectl top node
NAME          CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
k8s-master1   125m         6%     1140Mi          29%
k8s-node1     39m          1%     587Mi           15%
k8s-node2     39m          1%     479Mi           12%

7、 结果展示，在dashboard页面可以看到资源监控数据