k8s 安装ingress nginx controller 并部署.net core ingress服务

k8s 安装ingress nginx controller 并部署.net core ingress服务

本地k8s集群概览

192.168.28.132 k8smaster
192.168.28.133 k8snode1
192.168.28.134 k8snode2
192.168.28.135 k8snode3
192.168.28.131 www.img.com

www.img.com 是harbor 镜像仓库

k8s (kubectl version)版本是 1.23

准备安装ingress-nginx/controller-v1.1.0

参考：https://blog.csdn.net/qq_41586875/article/details/120896074，

参考（v1.0.0 的安装和使用）：https://www.cnblogs.com/cnsre/p/15218379.html

一、修改apiserver 启动参数

vi /etc/kubernetes/manifests/kube-apiserver.yaml

在“enable-admission-plugins=NodeRestriction”后面添加 “,MutatingAdmissionWebhook,ValidatingAdmissionWebhook” 参数

--enable-admission-plugins=NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook

二、下载ingress-nginx controller：

https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.0/deploy/static/provider/cloud/deploy.yaml

(下载不了，可能需要KX上网或绑定hosts)

复制一份，命名为“deploy-v1.1.0.yaml”，

并修改“deploy-v1.1.0.yaml”，

搜索Deployment （dnsPolicy）并修改，带“#”号的是修改点，

1. dnsPolicy换成ClusterFirstWithHostNet
2. 新加 hostNetwork: true
3. 新加 nodeName: k8snode1 ，指定部署到k8snode1
4. 新加

tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists

“deploy-v1.1.0.yaml” - Deployment 局部修改示例：

  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/component: controller
    spec:
      dnsPolicy: ClusterFirstWithHostNet  #既能使用宿主机DNS，又能使用集群DNS
      hostNetwork: true                   #与宿主机共享网络
      nodeName: k8snode1              #设置只能在k8snode1节点运行
      tolerations:  					  #设置能容忍master污点
      - key: node-role.kubernetes.io/master
        operator: Exists
      containers:
        - name: controller
          image: k8s.gcr.io/ingress-nginx/controller:v1.1.0@sha256:f766669fdcf3dc26347ed273a55e754b427eb4411ee075a53f30718b4499076a
          imagePullPolicy: IfNotPresent

三、安装ingress-nginx-controller：

kubectl apply -f deploy-v1.1.0.yaml

(镜像拉不下来时，可能需KX上网)

查看状态

kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx -o wide

NAME                                        READY   STATUS      RESTARTS   AGE   IP               NODE       NOMINATED NODE   READINESS GATES
ingress-nginx-admission-create-rnfvr        0/1     Completed   0          98m   10.244.3.67      k8snode3   <none>           <none>
ingress-nginx-admission-patch-mzsv2         0/1     Completed   0          98m   10.244.2.6       k8snode2   <none>           <none>
ingress-nginx-controller-648c6ccb64-lnstv   1/1     Running     0          98m   192.168.28.133   k8snode1   <none>           <none>

可以看到nginx-controller部署到了node1,内网IP是：192.168.28.133

如果一直是非“Running”，可以用下面的命令查看pod状态，然后百度解决

kubectl describe pod ingress-nginx-controller-648c6ccb64-lnstv --namespace=ingress-nginx

到node-1上看下本地端口：

netstat -lntup | grep nginx

tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      7385/nginx: worker
tcp        0      0 127.0.0.1:10245         0.0.0.0:*               LISTEN      55267/nginx-ingress
tcp        0      0 127.0.0.1:10246         0.0.0.0:*               LISTEN      7382/nginx: worker
tcp        0      0 127.0.0.1:10247         0.0.0.0:*               LISTEN      7382/nginx: worker
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      7385/nginx: worker
tcp        0      0 0.0.0.0:8181            0.0.0.0:*               LISTEN      7385/nginx: worker
tcp6       0      0 :::8443                 :::*                    LISTEN      55267/nginx-ingress
tcp6       0      0 :::443                  :::*                    LISTEN      7382/nginx: worker
tcp6       0      0 :::10254                :::*                    LISTEN      55267/nginx-ingress
tcp6       0      0 :::80                   :::*                    LISTEN      7382/nginx: worker
tcp6       0      0 :::8181                 :::*                    LISTEN      7382/nginx: worker

四、创建 ingress 服务：

参考： https://blog.csdn.net/qq_16538827/article/details/120265570

用到的.net core mvc Dockerfile等

打包.net core mvc 镜像

Dockerfile：

#添加.net6基础镜像
FROM mcr.microsoft.com/dotnet/aspnet:6.0
WORKDIR /app

# 一般情况下必须开放
EXPOSE 80
# 如果使用https，记得打开443端口，但是一般不用
#EXPOSE 443

#修改为上海时区
ENV TIME_ZONE Asia/Shanghai
RUN ln -snf /usr/share/zoneinfo/$TIME_ZONE /etc/localtime

COPY publish/ /app
ENTRYPOINT ["dotnet", "net6mvc.dll"]

k8s 上部署 .net core mvc 服务

web-pod.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: net6mvc
  labels:
    k8s-app: net6mvc
spec:
  replicas: 3
  selector:
    matchLabels:
      k8s-app: net6mvc
  template:
    metadata:
      labels:
        k8s-app: net6mvc
    spec:
      containers:
      - name: net6mvc
        image: www.img.com/library/net6-mvc:1.0
        imagePullPolicy: Always
        ports:
        - containerPort: 80
---
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: net6mvc
  name: net6mvc
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 80
    nodePort: 32143
  selector:
    k8s-app: net6mvc

k8s 上部署 ingress 服务

ingress-net6mvc.yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-net6mvc
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  rules:
  - host: www.net6mvc.com
    http:
      paths:
      - path: "/"
        pathType: Prefix
        backend:
          service:
            name: net6mvc
            port:
              number: 80

www.net6mvc.com 是随便起的域名。

docker build -t www.img.com/library/net6-mvc:1.0 .
docker push www.img.com/library/net6-mvc:1.0
docker rmi www.img.com/library/net6-mvc:1.0

kubectl apply -f web-pod.yaml

kubectl apply -f ingress-net6mvc.yaml

查看ingress-nginx 的内网工作端口

kubectl get svc -n ingress-nginx

[root@k8smaster home]# kubectl get svc -n ingress-nginx
NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.97.204.228    <pending>     80:32048/TCP,443:32094/TCP   115m
ingress-nginx-controller-admission   ClusterIP      10.108.100.124   <none>        443/TCP                      115m

可以看到nginx-controller http端口是：32048

C:\windows\system32\drivers\etc\，绑定hosts后，

192.168.28.133 www.net6mvc.com

浏览器访问：

http://www.net6mvc.com:32048/