k8s 安装ingress nginx controller 并部署.net core ingress服务

本地k8s集群概览

192.168.28.132 k8smaster

192.168.28.133 k8snode1

192.168.28.134 k8snode2

192.168.28.135 k8snode3

192.168.28.131 www.img.com

www.img.com 是harbor 镜像仓库

k8s (kubectl version)版本是 1.23

准备安装ingress-nginx/controller-v1.1.0

参考：https://blog.csdn.net/qq_41586875/article/details/120896074，

参考（v1.0.0 的安装和使用）：https://www.cnblogs.com/cnsre/p/15218379.html

一、修改apiserver 启动参数

vi /etc/kubernetes/manifests/kube-apiserver.yaml

在“enable-admission-plugins=NodeRestriction”后面添加 “,MutatingAdmissionWebhook,ValidatingAdmissionWebhook” 参数

--enable-admission-plugins=NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook

二、下载ingress-nginx controller：

https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.0/deploy/static/provider/cloud/deploy.yaml

(下载不了，可能需要KX上网或绑定hosts)

复制一份，命名为“deploy-v1.1.0.yaml”，

并修改“deploy-v1.1.0.yaml”，

搜索Deployment （dnsPolicy）并修改，带“#”号的是修改点，

dnsPolicy换成ClusterFirstWithHostNet
新加 hostNetwork: true
新加 nodeName: k8snode1 ，指定部署到k8snode1
新加

tolerations:

      - key: node-role.kubernetes.io/master

        operator: Exists

“deploy-v1.1.0.yaml” - Deployment 局部修改示例：

  template:

    metadata:

      labels:

        app.kubernetes.io/name: ingress-nginx

        app.kubernetes.io/instance: ingress-nginx

        app.kubernetes.io/component: controller

    spec:

      dnsPolicy: ClusterFirstWithHostNet  #既能使用宿主机DNS，又能使用集群DNS

      hostNetwork: true                   #与宿主机共享网络

      nodeName: k8snode1              #设置只能在k8snode1节点运行

      tolerations:  					  #设置能容忍master污点

      - key: node-role.kubernetes.io/master

        operator: Exists

      containers:

        - name: controller

          image: k8s.gcr.io/ingress-nginx/controller:v1.1.0@sha256:f766669fdcf3dc26347ed273a55e754b427eb4411ee075a53f30718b4499076a

          imagePullPolicy: IfNotPresent

三、安装ingress-nginx-controller：

kubectl apply -f deploy-v1.1.0.yaml

(镜像拉不下来时，可能需KX上网)

查看状态

kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx -o wide

NAME                                        READY   STATUS      RESTARTS   AGE   IP               NODE       NOMINATED NODE   READINESS GATES

ingress-nginx-admission-create-rnfvr        0/1     Completed   0          98m   10.244.3.67      k8snode3   <none>           <none>

ingress-nginx-admission-patch-mzsv2         0/1     Completed   0          98m   10.244.2.6       k8snode2   <none>           <none>

ingress-nginx-controller-648c6ccb64-lnstv   1/1     Running     0          98m   192.168.28.133   k8snode1   <none>           <none>

可以看到nginx-controller部署到了node1,内网IP是：192.168.28.133

如果一直是非“Running”，可以用下面的命令查看pod状态，然后百度解决

kubectl describe pod ingress-nginx-controller-648c6ccb64-lnstv --namespace=ingress-nginx

到node-1上看下本地端口：

netstat -lntup | grep nginx

tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      7385/nginx: worker

tcp        0      0 127.0.0.1:10245         0.0.0.0:*               LISTEN      55267/nginx-ingress

tcp        0      0 127.0.0.1:10246         0.0.0.0:*               LISTEN      7382/nginx: worker

tcp        0      0 127.0.0.1:10247         0.0.0.0:*               LISTEN      7382/nginx: worker

tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      7385/nginx: worker

tcp        0      0 0.0.0.0:8181            0.0.0.0:*               LISTEN      7385/nginx: worker

tcp6       0      0 :::8443                 :::*                    LISTEN      55267/nginx-ingress

tcp6       0      0 :::443                  :::*                    LISTEN      7382/nginx: worker

tcp6       0      0 :::10254                :::*                    LISTEN      55267/nginx-ingress

tcp6       0      0 :::80                   :::*                    LISTEN      7382/nginx: worker

tcp6       0      0 :::8181                 :::*                    LISTEN      7382/nginx: worker

四、创建 ingress 服务：

参考： https://blog.csdn.net/qq_16538827/article/details/120265570

用到的.net core mvc Dockerfile等

打包.net core mvc 镜像

Dockerfile：

#添加.net6基础镜像

FROM mcr.microsoft.com/dotnet/aspnet:6.0

WORKDIR /app

# 一般情况下必须开放

EXPOSE 80

# 如果使用https，记得打开443端口，但是一般不用

#EXPOSE 443

#修改为上海时区

ENV TIME_ZONE Asia/Shanghai

RUN ln -snf /usr/share/zoneinfo/$TIME_ZONE /etc/localtime

COPY publish/ /app

ENTRYPOINT ["dotnet", "net6mvc.dll"]

k8s 上部署 .net core mvc 服务

web-pod.yaml:

apiVersion: apps/v1

kind: Deployment

metadata:

  name: net6mvc

  labels:

    k8s-app: net6mvc

spec:

  replicas: 3

  selector:

    matchLabels:

      k8s-app: net6mvc

  template:

    metadata:

      labels:

        k8s-app: net6mvc

    spec:

      containers:

      - name: net6mvc

        image: www.img.com/library/net6-mvc:1.0

        imagePullPolicy: Always

        ports:

        - containerPort: 80

---

kind: Service

apiVersion: v1

metadata:

  labels:

    k8s-app: net6mvc

  name: net6mvc

spec:

  type: NodePort

  ports:

  - port: 80

    targetPort: 80

    nodePort: 32143

  selector:

    k8s-app: net6mvc

k8s 上部署 ingress 服务

ingress-net6mvc.yaml:

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: ingress-net6mvc

  annotations:

    kubernetes.io/ingress.class: "nginx"

    nginx.ingress.kubernetes.io/use-regex: "true"

spec:

  rules:

  - host: www.net6mvc.com

    http:

      paths:

      - path: "/"

        pathType: Prefix

        backend:

          service:

            name: net6mvc

            port:

              number: 80

www.net6mvc.com 是随便起的域名。

docker build -t www.img.com/library/net6-mvc:1.0 .

docker push www.img.com/library/net6-mvc:1.0

docker rmi www.img.com/library/net6-mvc:1.0

kubectl apply -f web-pod.yaml

kubectl apply -f ingress-net6mvc.yaml

查看ingress-nginx 的内网工作端口

kubectl get svc -n ingress-nginx

[root@k8smaster home]# kubectl get svc -n ingress-nginx

NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE

ingress-nginx-controller             LoadBalancer   10.97.204.228    <pending>     80:32048/TCP,443:32094/TCP   115m

ingress-nginx-controller-admission   ClusterIP      10.108.100.124   <none>        443/TCP                      115m

可以看到nginx-controller http端口是：32048

C:\windows\system32\drivers\etc\，绑定hosts后，

192.168.28.133 www.net6mvc.com

浏览器访问：

http://www.net6mvc.com:32048/