Kubernetes Heapster

Heapster是容器集群监控和性能分析工具，HPA、Dashborad、Kubectl top都依赖于heapster收集的数据。

但是Heapster从kubernetes 1.8以后已经被遗弃了...... 被metrics-server所替代......

kubernetes 1.11 中部署Heapster 1.5.4版本的过程

Heapster 部署yaml文件

apiVersion: v1

kind: ServiceAccount

metadata:

  name: heapster

  namespace: kube-system

---

kind: ClusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1

metadata:

  name: heapster

roleRef:

  apiGroup: rbac.authorization.k8s.io

  kind: ClusterRole

  name: system:heapster

subjects:

- kind: ServiceAccount

  name: heapster

  namespace: kube-system

---

apiVersion: apps/v1

kind: Deployment

metadata:

  name: heapster

  namespace: kube-system

spec:

  replicas: 1

  selector:

      matchLabels:

        k8s-app: heapster

  template:

    metadata:

      labels:

        task: monitoring

        k8s-app: heapster

    spec:

      serviceAccountName: heapster

      containers:

      - name: heapster

        # image: k8s.gcr.io/heapster-amd64:v1.5.4 将默认google的官方镜像替换为阿里云镜像，否则你懂得

        image: registry.cn-hangzhou.aliyuncs.com/google_containers/heapster-amd64:v1.5.4

        command:

        - /heapster

        - --source=kubernetes:https://kubernetes.default?useServiceAccount=true&kubeletHttps=true&kubeletPort=10250&insecure=true

---

apiVersion: v1

kind: Service

metadata:

  labels:

    task: monitoring

    # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)

    # If you are NOT using this as an add-on, you should comment out this line.

    kubernetes.io/cluster-service: 'true'

    kubernetes.io/name: Heapster

  name: heapster

  namespace: kube-system

spec:

  ports:

  - port: 80

    targetPort: 8082

  selector:

    k8s-app: heapster

heapster.yaml

heapster 启动参数说明：

inClusterConfig - Use kube config in service accounts associated with Heapster's namespace. (default: true)
kubeletPort - kubelet port to use (default: 10255)
kubeletHttps - whether to use https to connect to kubelets (default: false)
insecure - whether to trust Kubernetes certificates (default: false)
auth - client auth file to use. Set auth if the service accounts are not usable.
useServiceAccount - whether to use the service account token if one is mounted at /var/run/secrets/kubernetes.io/serviceaccount/token (default: false)

使用： kubectl apply -f heapster.yaml 部署

部署后查看log日志，发现一直提示“403 Forbidden”, response: "Forbidden (user=system:serviceaccount:kube-system:heapster, verb=create, resource=nodes, subresource=stats)"

[root@node01 heapster-yaml]# kubectl logs --namespace=kube-system heapster-868d5cd7f-855k7

I0912 14:36:48.175115       1 heapster.go:78] /heapster --source=kubernetes:https://kubernetes.default?useServiceAccount=true&kubeletHttps=true&kubeletPort=10250&insecure=true

I0912 14:36:48.175174       1 heapster.go:79] Heapster version v1.5.4

I0912 14:36:48.175366       1 configs.go:61] Using Kubernetes client with master "https://kubernetes.default" and version v1

I0912 14:36:48.175393       1 configs.go:62] Using kubelet port 10250

I0912 14:36:48.185828       1 heapster.go:202] Starting with Metric Sink

I0912 14:36:48.199515       1 heapster.go:112] Starting heapster on port 8082

E0912 14:37:05.000327       1 kubelet.go:288] node node01 is not ready

E0912 14:38:05.016044       1 manager.go:101] Error in scraping containers from kubelet:172.16.65.181:10250: failed to get all container stats from Kubelet URL "https://172.16.65.181:10250/stats/container/": request failed - "403 Forbidden", response: "Forbidden (user=system:serviceaccount:kube-system:heapster, verb=create, resource=nodes, subresource=stats)"

W0912 14:38:25.000643       1 manager.go:152] Failed to get all responses in time (got 0/1)

E0912 14:39:05.008534       1 manager.go:101] Error in scraping containers from kubelet:172.16.65.181:10250: failed to get all container stats from Kubelet URL "https://172.16.65.181:10250/stats/container/": request failed - "403 Forbidden", response: "Forbidden (user=system:serviceaccount:kube-system:heapster, verb=create, resource=nodes, subresource=stats)"

W0912 14:39:25.000674       1 manager.go:152] Failed to get all responses in time (got 0/1)

E0912 14:40:05.009955       1 manager.go:101] Error in scraping containers from kubelet:172.16.65.181:10250: failed to get all container stats from Kubelet URL "https://172.16.65.181:10250/stats/container/": request failed - "403 Forbidden", response: "Forbidden (user=system:serviceaccount:kube-system:heapster, verb=create, resource=nodes, subresource=stats)"

W0912 14:40:25.001231       1 manager.go:152] Failed to get all responses in time (got 0/1)

E0912 14:41:05.017198       1 manager.go:101] Error in scraping containers from kubelet:172.16.65.181:10250: failed to get all container stats from Kubelet URL "https://172.16.65.181:10250/stats/container/": request failed - "403 Forbidden", response: "Forbidden (user=system:serviceaccount:kube-system:heapster, verb=create, resource=nodes, subresource=stats)"

W0912 14:41:25.000846       1 manager.go:152] Failed to get all responses in time (got 0/1)

查看ClusterRole: system:heapster的权限，发现的确没有针对Resource: nodes/stats 的create权限

[root@node01 heapster-yaml]# kubectl describe clusterrole system:heapster

Name:         system:heapster

Labels:       kubernetes.io/bootstrapping=rbac-defaults

Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"rbac.authorization.k8s.io/v1","kind":"ClusterRole","metadata":{"annotations":{"rbac.authorization.kubernetes.io/autoupdate":"true"},"lab...

              rbac.authorization.kubernetes.io/autoupdate=true

PolicyRule:

  Resources               Non-Resource URLs  Resource Names  Verbs

  ---------               -----------------  --------------  -----

  events                  []                 []              [get list watch]

  namespaces              []                 []              [get list watch]

  nodes                   []                 []              [get list watch]

  pods                    []                 []              [get list watch]

  deployments.extensions  []                 []              [get list watch]

修改ClusterRole: system:heapster的权限:

1. 查看system:heapster yaml格式, 保存为 heapster-clusterrole.yaml

yaml

[root@node01 heapster-yaml]# kubectl get clusterrole system:heapster -o yaml

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRole

metadata:

  annotations:

    kubectl.kubernetes.io/last-applied-configuration: |

      {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"ClusterRole","metadata":{"annotations":{"rbac.authorization.kubernetes.io/autoupdate":"true"},"labels":{"kubernetes.io/bootstrapping":"rbac-defaults"},"name":"system:heapster","namespace":""},"rules":[{"apiGroups":[""],"resources":["events","namespaces","nodes","pods","nodes/stats"],"verbs":["create","get","list","watch"]},{"apiGroups":["extensions"],"resources":["deployments"],"verbs":["get","list","watch"]}]}

    rbac.authorization.kubernetes.io/autoupdate: "true"

  creationTimestamp: 2018-08-26T02:26:14Z

  labels:

    kubernetes.io/bootstrapping: rbac-defaults

  name: system:heapster

  resourceVersion: ""

  selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/system%3Aheapster

  uid: 67ef3689-a8d7-11e8-a891-000c29b52823

rules:

- apiGroups:

  - ""

  resources:

  - events

  - namespaces

  - nodes

  - pods

  verbs:

  - get

  - list

  - watch

- apiGroups:

  - extensions

  resources:

  - deployments

  verbs:

  - get

  - list

  - watch

2. 添加Resource: nodes/stats的create权限，并执行 kubectl apply -f heapster-clusterrole.yaml

yaml

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRole

metadata:

  annotations:

    rbac.authorization.kubernetes.io/autoupdate: "true"

  labels:

    kubernetes.io/bootstrapping: rbac-defaults

  name: system:heapster

rules:

- apiGroups:

  - ""

  resources:

  - events

  - namespaces

  - nodes

  - pods

  - nodes/stats

  verbs:

  - create

  - get

  - list

  - watch

- apiGroups:

  - extensions

  resources:

  - deployments

  verbs:

  - get

  - list

  - watch

3. 删除heapster重新部署

　　kubectl delete -f heapster.yaml

　　kubectl apply -f heapster.yaml

4. 重新部署后，查看log没有任何报错了

[root@node01 heapster-yaml]# kubectl logs --namespace=kube-system heapster-868d5cd7f-8zgxq

I0912 15:02:38.926068       1 heapster.go:78] /heapster --source=kubernetes:https://kubernetes.default?useServiceAccount=true&kubeletHttps=true&kubeletPort=10250&insecure=true

I0912 15:02:38.926128       1 heapster.go:79] Heapster version v1.5.4

I0912 15:02:38.926565       1 configs.go:61] Using Kubernetes client with master "https://kubernetes.default" and version v1

I0912 15:02:38.926647       1 configs.go:62] Using kubelet port 10250

I0912 15:02:38.938333       1 heapster.go:202] Starting with Metric Sink

I0912 15:02:38.949215       1 heapster.go:112] Starting heapster on port 8082

5. 功能测试

部署前：

[root@node01 heapster-yaml]# kubectl top pod

Error from server (NotFound): the server could not find the requested resource (get services http:heapster:)

部署后：

[root@node01 tomcat-mysql-yaml]# kubectl top pod

NAME                      CPU(cores)   MEMORY(bytes)

lxcfs-j8bpd               0m           4Mi

mysql-85c85c5668-fr6j7    0m           458Mi

tomcat-78c9778858-424gc   1m           93Mi

tomcat-78c9778858-42xz4   1m           97Mi

tomcat-78c9778858-6t7cd   1m           93Mi

Kubernetes Heapster的更多相关文章

详解k8s一个完整的监控方案(Heapster+Grafana+InfluxDB) - kubernetes
1.浅析整个监控流程 heapster以k8s内置的cAdvisor作为数据源收集集群信息,并汇总出有价值的性能数据(Metrics):cpu.内存.网络流量等,然后将这些数据输出到外部存储,如Inf ...
详解k8s原生的集群监控方案(Heapster+InfluxDB+Grafana) - kubernetes
1.浅析监控方案 heapster是一个监控计算.存储.网络等集群资源的工具,以k8s内置的cAdvisor作为数据源收集集群信息,并汇总出有价值的性能数据(Metrics):cpu.内存.netwo ...
使用 kubeadm 安装部署 kubernetes 1.9-部署heapster插件
1.先到外网下载好镜像倒进各个节点 2.下载yaml文件和创建应用 mkdir -p ~/k8s/heapster cd ~/k8s/heapster wget https://raw.githubu ...
Kubernetes 1.5集成heapster
Heapster是kubernetes集群监控工具.在1.2的时候,kubernetes的监控需要在node节点上运行cAdvisor作为agent收集本机和容器的资源数据,包括cpu.内存.网络.文 ...
Kubernetes监控:部署Heapster、InfluxDB和Grafana
本节内容: Kubernetes 监控方案 Heapster.InfluxDB和Grafana介绍安装配置Heapster.InfluxDB和Grafana 访问 grafana 访问 influx ...
Kubernetes dashboard集成heapster
图形化展示度量指标的实现需要集成k8s的另外一个Addons组件: Heapster . Heapster原生支持K8s(v1.0.6及以后版本)和 CoreOS ,并且支持多种存储后端,比如: In ...
kubernetes 监控方案之：heapster+influxdb+grafana(十八)
目录一.Heapster 介绍二.部署三.使用 heapster 已经 deprecated 了:https://github.com/kubernetes/heapster,所以下面的演示主要 ...
Openstack+Kubernetes+Docker微服务实践之路--Kubernetes
经过几番折腾终于搞定Kubernetes了,我们要在Openstack上部署Kubernetes集群,使用最新工具Kubeadm来安装,由于不能直接访问Kubernetes的源,我们需要一台可以穿墙的 ...
kubernetes组件
kubernetes组件 @(马克飞象)[k8s] 组件 kubernetes除了必备的dns和网络组件外,官方推出大量的cluster-monitoring,dashboard,fluentd-el ...

随机推荐

[Python2.x] 利用commands模块执行Linux shell命令
用Python写运维脚本时,经常需要执行linux shell的命令,Python中的commands模块专门用于调用Linux shell命令,并返回状态和结果,下面是commands模块的3个主要 ...
PDO drivers no value in Windows
学习php编程遇到 Uncaught exception 'PDOException' with message 'could not find driver' 或者 Undefined class ...
Codeforces Beta Round #25 (Div. 2)--A. IQ test
IQ test time limit per test 2 seconds memory limit per test 256 megabytes input standard input outpu ...
【转】NPOI自定义单元格背景颜色
经常在NPOI群里聊天时发现有人在问NPOI设置单元格背景颜色的问题,而Tony Qu大神的博客里没有相关教程,刚好最近在做项目时研究了一下这一块,在这里总结一下. 在NPOI中默认的颜色类是HSSF ...
django database relations
注意Django的生成的默认api from django.db import models class Place(models.Model): ''' pass class Restaurant( ...
移动端H5页面自适应手机屏幕宽度
1.由于本人使用的是sublime.text,使用rem就可以达到效果. 点击菜单中的preferences下的browse packages,选择cssrem-master,添加或者编写cssrem ...
extract
w http://php.net/manual/en/function.extract.php <?php /* Suppose that $var_array is an array retu ...
ehcache 的HelloWorld实现（二）
EhCache最新版本是3.X 本人一直用的是2.X 比较稳定功能足够用: 所以还是用2.X版本: 我们新建一个Maven项目, pom.xml里引入 ehcache支持: <dependen ...
【原创】学习CGLIB动态代理中遇到的问题
代码清单1 CGLIB动态代理 package wulj.proxy.cglibProxy; import java.lang.reflect.Method; import net.sf.cglib. ...
javascript之冒泡排序
Array.prototype.bubble = function () { var temp; //第1轮比较求第一,第二轮求第二,以此类推(i<=数组成员的个数); for (var i = ...

Kubernetes Heapster

kubernetes 1.11 中部署Heapster 1.5.4版本的过程

Kubernetes Heapster的更多相关文章

随机推荐

热门专题