问题

通过minikube启动k8s集群后，我们想安装启动k8s的dashboard插件，执行minikube dashboard后，一直卡住直到提示验证失败。

查看pod状态，我们看到kubernetes-dashboard下的2个pod实例的状态都是ImagePullBackOff

[dev ~]# kubectl get pod --all-namespaces

NAMESPACE              NAME                                         READY   STATUS             RESTARTS       AGE

kube-system            coredns-76fccbbb6b-wbx9v                     1/1     Running            0              110m

kube-system            etcd-minikube                                1/1     Running            0              110m

kube-system            kube-apiserver-minikube                      1/1     Running            0              110m

kube-system            kube-controller-manager-minikube             1/1     Running            0              110m

kube-system            kube-proxy-grk7h                             1/1     Running            0              110m

kube-system            kube-scheduler-minikube                      1/1     Running            0              110m

kube-system            storage-provisioner                          1/1     Running            1 (110m ago)   110m

kubernetes-dashboard   dashboard-metrics-scraper-5d59dccf9b-pzt8r   0/1     ImagePullBackOff   0              108m

kubernetes-dashboard   kubernetes-dashboard-7779f9b69b-zz2wm        0/1     ImagePullBackOff   0              108m

我们查看具体的失败原因可以看到：

查看 dashboard-metrics-scraper 失败原因

[dev ~]# kubectl describe pod dashboard-metrics-scraper-5d59dccf9b-pzt8r --namespace=kubernetes-dashboard

Events:

  Type     Reason   Age                     From     Message

  ----     ------   ----                    ----     -------

  Normal   Pulling  34m (x19 over 109m)     kubelet  Pulling image "docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c"

  Warning  Failed   24m (x21 over 109m)     kubelet  Failed to pull image "docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c": Error response from daemon: Get "https://registry-1.docker.io/v2/": dial tcp: lookup registry-1.docker.io on 192.168.49.1:53: server misbehaving

  Normal   BackOff  4m50s (x442 over 109m)  kubelet  Back-off pulling image "docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c"

  Warning  Failed   4m37s (x443 over 109m)  kubelet  Error: ImagePullBackOff

查看 kubernetes-dashboard 失败原因

[dev ~]# kubectl describe pod kubernetes-dashboard-7779f9b69b-zz2wm --namespace=kubernetes-dashboard

Events:

  Type     Reason   Age                   From     Message

  ----     ------   ----                  ----     -------

  Normal   Pulling  36m (x20 over 116m)   kubelet  Pulling image "docker.io/kubernetesui/dashboard:v2.7.0@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93"

  Warning  Failed   31m (x21 over 116m)   kubelet  Failed to pull image "docker.io/kubernetesui/dashboard:v2.7.0@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93": Error response from daemon: Get "https://registry-1.docker.io/v2/": dial tcp: lookup registry-1.docker.io on 192.168.49.1:53: server misbehaving

  Normal   BackOff  96s (x486 over 116m)  kubelet  Back-off pulling image "docker.io/kubernetesui/dashboard:v2.7.0@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93"

  Warning  Failed   82s (x487 over 116m)  kubelet  Error: ImagePullBackOff

2个pod实例启动失败原因都是dashboard所需镜像pull失败导致。

解决办法

手动pull镜像，并设置tag

拉取失败的总共有2个镜像：

docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c
docker.io/kubernetesui/dashboard:v2.7.0@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93

因为pull镜像失败的网络问题无法解决，我们只能pull国内镜像，并给镜像打上原始镜像名称的标记。

进入minikube的环境

我们需要先进入minikube的环境，如果直接在host环境中拉取镜像，minikube是获取不到的。

minikube ssh

进入环境后我们可以执行 docker image ls -a查看当前环境的镜像列表，其中没有包含所缺失的镜像。

拉取国内镜像，并打上原始镜像的tag

# 手动pull metrics-scraper:v1.0.8

docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-scraper:v1.0.8

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-scraper:v1.0.8 docker.io/kubernetesui/metrics-scraper:v1.0.8

# 手动pull dashboard:v2.7.0

docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/dashboard:v2.7.0

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/dashboard:v2.7.0 docker.io/kubernetesui/dashboard:v2.7.0

查看现有镜像

[dev ~]# minikube ssh

docker@minikube:~$ docker image ls -a

REPOSITORY                                                                    TAG        IMAGE ID       CREATED         SIZE

registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver            v1.32.0    c2e17b8d0f4a   5 months ago    97MB

registry.k8s.io/kube-apiserver                                                v1.32.0    c2e17b8d0f4a   5 months ago    97MB

registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager   v1.32.0    8cab3d2a8bd0   5 months ago    89.7MB

registry.k8s.io/kube-controller-manager                                       v1.32.0    8cab3d2a8bd0   5 months ago    89.7MB

registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler            v1.32.0    a389e107f4ff   5 months ago    69.6MB

registry.k8s.io/kube-scheduler                                                v1.32.0    a389e107f4ff   5 months ago    69.6MB

registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy                v1.32.0    040f9f8aac8c   5 months ago    94MB

registry.k8s.io/kube-proxy                                                    v1.32.0    040f9f8aac8c   5 months ago    94MB

registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen      v1.4.4     a62eeff05ba5   7 months ago    63.8MB

registry.cn-hangzhou.aliyuncs.com/google_containers/etcd                      3.5.16-0   a9e7e6b294ba   8 months ago    150MB

registry.k8s.io/etcd                                                          3.5.16-0   a9e7e6b294ba   8 months ago    150MB

registry.cn-hangzhou.aliyuncs.com/google_containers/coredns                   v1.11.3    c69fa2e9cbf5   9 months ago    61.8MB

registry.k8s.io/coredns/coredns                                               v1.11.3    c69fa2e9cbf5   9 months ago    61.8MB

registry.cn-hangzhou.aliyuncs.com/google_containers/pause                     3.10       873ed7510279   12 months ago   736kB

registry.k8s.io/pause                                                         3.10       873ed7510279   12 months ago   736kB

kubernetesui/dashboard                                                        v2.7.0     07655ddf2eeb   2 years ago     246MB

registry.cn-hangzhou.aliyuncs.com/google_containers/dashboard                 v2.7.0     07655ddf2eeb   2 years ago     246MB

registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-scraper           v1.0.8     115053965e86   2 years ago     43.8MB

kubernetesui/metrics-scraper                                                  v1.0.8     115053965e86   2 years ago     43.8MB

gcr.io/k8s-minikube/storage-provisioner                                       v5         6e38f40d628d   4 years ago     31.5MB

registry.cn-hangzhou.aliyuncs.com/google_containers/storage-provisioner       v5         6e38f40d628d   4 years ago     31.5MB

缺失的镜像列表都弄好了。

停掉当前的dashboard

这个步骤可以通过kubectl在外部删掉启动失败的2个pod，或者在minikube内部手动运行kubectl删掉都可以。

在外部删掉

kubectl delete pod dashboard-metrics-scraper-5d59dccf9b-pzt8r --namespace kubernetes-dashboard

kubectl delete pod kubernetes-dashboard-7779f9b69b-zz2wm --namespace kubernetes-dashboard

进入minikube环境，手动运行kubectl 删掉dashboard的pod

[dev ~]# minikube ssh

docker@minikube:~$ sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.0/kubectl delete -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml

namespace "kubernetes-dashboard" deleted

clusterrole.rbac.authorization.k8s.io "kubernetes-dashboard" deleted

clusterrolebinding.rbac.authorization.k8s.io "kubernetes-dashboard" deleted

configmap "kubernetes-dashboard-settings" deleted

deployment.apps "dashboard-metrics-scraper" deleted

deployment.apps "kubernetes-dashboard" deleted

role.rbac.authorization.k8s.io "kubernetes-dashboard" deleted

rolebinding.rbac.authorization.k8s.io "kubernetes-dashboard" deleted

serviceaccount "kubernetes-dashboard" deleted

secret "kubernetes-dashboard-certs" deleted

secret "kubernetes-dashboard-csrf" deleted

secret "kubernetes-dashboard-key-holder" deleted

service "kubernetes-dashboard" deleted

service "dashboard-metrics-scraper" deleted

编辑dashboard的deployment配置文件，修改其中的镜像地址

我们需要先进入minikube环境

minikube ssh

查看配置

sudo cat /etc/kubernetes/addons/dashboard-dp.yaml

配置太长了，只截取关注的这部分：

kind: Deployment

apiVersion: apps/v1

spec:

    spec:

      containers:

        - name: dashboard-metrics-scraper

          image: docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c

---

kind: Deployment

apiVersion: apps/v1

spec:

    spec:

      containers:

        - name: kubernetes-dashboard

          # WARNING: This must match pkg/minikube/bootstrapper/images/images.go

          image: docker.io/kubernetesui/dashboard:v2.7.0@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93

          ports:

            - containerPort: 9090

              protocol: TCP

可以看到，pull失败的镜像地址，就是在这个配置文件里配置，接下来，我们需要修改这个配置文件，将image地址中的sha256删掉，并添加镜像pull规则：

    spec:

      containers:

        - name: dashboard-metrics-scraper

          image: docker.io/kubernetesui/metrics-scraper:v1.0.8

          imagePullPolicy: IfNotPresent

----

    spec:

      containers:

        - name: kubernetes-dashboard

          # WARNING: This must match pkg/minikube/bootstrapper/images/images.go

          image: docker.io/kubernetesui/dashboard:v2.7.0

          imagePullPolicy: IfNotPresent

主要是2个字段：

image 将@和后面的字符串删掉，因为我们国内的镜像这个sha256和配置中的看起来并不一样。
imagePullPolicy 设置为 IfNotPresent，表示镜像存在就不会再pull一次。

手动启动dashboard

docker@minikube:~$ sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml

namespace/kubernetes-dashboard created

clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created

clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created

configmap/kubernetes-dashboard-settings created

Warning: spec.template.metadata.annotations[seccomp.security.alpha.kubernetes.io/pod]: non-functional in v1.27+; use the "seccompProfile" field instead

deployment.apps/dashboard-metrics-scraper created

deployment.apps/kubernetes-dashboard created

role.rbac.authorization.k8s.io/kubernetes-dashboard created

rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created

serviceaccount/kubernetes-dashboard created

secret/kubernetes-dashboard-certs created

secret/kubernetes-dashboard-csrf created

secret/kubernetes-dashboard-key-holder created

service/kubernetes-dashboard created

service/dashboard-metrics-scraper created

然后退出minikube环境，查看dashboard的pod运行状态：

[dev ~]# kubectl get pod -A

NAMESPACE              NAME                                          READY   STATUS              RESTARTS         AGE

kube-system            coredns-76fccbbb6b-vtdbf                      1/1     Running             5 (12m ago)      13h

kube-system            etcd-minikube                                 1/1     Running             10 (12m ago)     2d

kube-system            kube-apiserver-minikube                       1/1     Running             60 (13m ago)     2d

kube-system            kube-controller-manager-minikube              1/1     Running             34 (9m42s ago)   2d

kube-system            kube-proxy-4n788                              1/1     Running             1 (12m ago)      2d

kube-system            kube-scheduler-minikube                       1/1     Running             12 (12m ago)     2d

kubernetes-dashboard   dashboard-metrics-scraper-545b5664b6-mcrzh    1/1     Running             0                5m27s

kubernetes-dashboard   kubernetes-dashboard-7f67ccd984-ql4cn         1/1     Running             0                5m27s

minikube dashboard ImagePullBackOff 失败问题的解决方法的更多相关文章

phpStudy启动失败时的解决方法
phpStudy启动失败时的解决方法 phpStudy启动失败,原因一是防火墙拦截,二是80端口已经被别的程序占用,如IIS,迅雷等:三是没有安装VC9运行库,php和apache都是VC9编译.解决 ...
mysql创建外链失败1005错误解决方法
mysql创建外链失败1005错误解决方法错误号:1005错误信息:Can't create table 'webDB.#sql-397_61df' (errno: 150)解决方法错误原因有四: ...
【常见的SQL Server连接失败错误以及解决方法】
[常见的SQL Server连接失败错误以及解决方法] http://blog.csdn.net/feixianxxx/article/details/5523922 ADO连接SQL Server ...
pip安装包失败的常见解决方法
pip安装包失败的常见解决方法方法一当安装包失效的时候可以去网站下载whl文件.下载的时候应该先查看电脑上 python 的版本. 地址:https://www.lfd.uci.edu/~gohl ...
“访问 IIS 元数据库失败”错误的解决方法
1.依次点击“开始”-“运行”. 2.在“运行”栏内输入 “C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis.exe -i ...
hanlp在Python环境中的安装失败后的解决方法
Hanlp是由一系列模型与算法组成的javag工具包,目标是普及自然语言处理再生环境中的应用.有很多人在安装hanlp的时候会遇到安装失败的情况,下面就是某大神的分享的在python环境中安装失败的解 ...
【Gradle】配置中引用的jar包版本后面自动加冒号导致引入jar包失败的问题/gradle中引用jar包版本不一致的问题/gradle中引用jar失败的问题解决方法
idea中 gradle中引用jar包,版本后面默认加:的问题 gradle中引用jar包版本不一致的问题 gradle中引用jar失败的问题如上题目所示,三个问题其实都是同一样的简单又恶心,因为 ...
安装VMTools失败的三类解决方法（Windows、Linux、MacOs）
前言写这篇笔记的原因,是前几天在虚拟机 Vmware 中重新安装了几个操作系统,突然发现 VMTools 这个工具成了一个特殊的问题,以前还没有发现,因为通常它就给你自动安装了.但是大多数时候也是需 ...
BurpSuite下提示embedded browser initialisation failed（嵌入式浏览器初始化失败）的解决方法
BurpSuite可谓是渗透测试过程经常使用的神器之一,但使用中经常会碰到奇奇怪怪的问题,比如有时抓http包,发送到Repeater(中继器,也叫重发器)模块后,在右边Render模块下,却无法看到 ...
wordpress定时发布失败Missed Schedule解决方法
今天一位客户的网站设置了文章定时发布但却没有发布成功,提示Missed Schedule错误,把所有的插件都暂停了再设置定时发布也不行,那就只能寻找其他的解决方案了.ytkah查看一下有没相关修复插件 ...

随机推荐

「六」Goaccess实现可视化
下载 apt install goaccess 使用goaccess进行监控 LANG="en_US.UTF-8" bash -c 'goaccess logs/access.lo ...
【面试题】实现 queryParse 函数，完成解析 URL 参数的功能
问题:实现 queryParse 函数,完成解析 URL 参数的功能 /** * 问题:实现 queryParse 函数,完成解析 URL 参数的功能 * * 用法: * ```js * const ...
nginx配置2个不同端口的应用
如何配置nginx,在同一台服务器上,部署2个不同端口的web应用? 1,利用Django框架搭建的web应用,默认端口是8000: 2,利用Flask框架搭建的web应用,默认端口是5000: 第一 ...
Ollama系列05：Ollama API 使用指南
本文是Ollama系列教程的第5篇,在前面的4篇内容中,给大家分享了如何再本地通过Ollama运行DeepSeek等大模型,演示了chatbox.CherryStudio等UI界面中集成Ollama的 ...
Tomcat性能优化以及 jvm 参数设置
linux ps 命令的结果中 VSZ,RSS,STAT 的含义和大小参数名含义单位 USER 进程所属用户 PID 进程ID %CPU 进程占用CPU百分比 %MEM 进程占用内存 ...
万字长文详解SIFT特征提取
本文对 SIFT 算法进行了详细梳理.SIFT即尺度不变特征变换(Scale-Invariant Feature Transform),是一种用于检测和描述图像局部特征的算法.该算法对图像的尺度和旋转 ...
CSAPP学习笔记——Chapter12 并行编程
CSAPP学习笔记--Chapter12 并行编程并发编程有着其独特的魅力,之前接触cuda编程的时候,感受到一些,没想到书里还有相关的内容.今天我们主要围绕进程,I/O多路复用,线程三种并发的方式 ...
继承中成员变量和成员方法的访问特点-java se进阶篇 day01
1.继承中成员变量的访问特点 1.成员变量重名如图父类中有age变量,子类中也有age变量,这时打印age,出现的是10还是20呢? 答:根据就近原则,出现的是20 2.使用父类成员变量--sup ...
【JDBC第9章】Apache-DBUtils实现CRUD操作
第9章:Apache-DBUtils实现CRUD操作 9.1 Apache-DBUtils简介 commons-dbutils 是 Apache 组织提供的一个开源 JDBC工具类库,它是对JDBC的 ...
第10章面向对象编程(高级部分)-cnblog
类变量与类方法 static修饰的成员变量(类变量,静态变量)的特性? 同一个类所有对象共享类变量是随着类的加载而创建, 所以即使没有创建对象实例也可以访问 ,但是类变量的访问, 必须遵守相关的访 ...

minikube dashboard ImagePullBackOff 失败问题的解决方法

问题