kubernetes 安装 ingress controller

ingress-nginx

ingress 官方网站

ingress 仓库地址

ingress-nginx v1.0 最新版本 v1.0

适用于 Kubernetes 版本 v1.19+ （包括 v1.19 ）

Kubernetes-v1.22+ 需要使用 ingress-nginx>=1.0，因为 networking.k8s.io/v1beta 已经移除

直接部署 ingress-nginx

直接部署比较简单，直接拉去 girhub 的文件就可以了，如果遇到长时间无响应，可以终止任务从新拉取。

拉取镜像部分，可以修改为一下的镜像地址

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.0/deploy/static/provider/baremetal/deploy.yaml

sed -i 's@k8s.gcr.io/ingress-nginx/controller:v1.0.0\(.*\)@willdockerhub/ingress-nginx-controller:v1.0.0@' deploy.yaml

sed -i 's@k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0\(.*\)$@hzde0128/kube-webhook-certgen:v1.0@' deploy.yaml

kubectl apply -f ingress-nginx.yaml

检查安装

Completed 状态的是正常的，可以忽略。

[root@master ~]# kubectl get po -n ingress-nginx

NAME                                        READY   STATUS      RESTARTS   AGE

ingress-nginx-admission-create-pm6sw        0/1     Completed   0          22m

ingress-nginx-admission-patch-m8w94         0/1     Completed   0          22m

ingress-nginx-controller-7d4df87d89-272ft   1/1     Running     0          22m

[root@master ~]# kubectl get svc -n ingress-nginx

NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE

ingress-nginx-controller             NodePort    10.96.88.139   <none>        80:30497/TCP,443:32581/TCP   22m

ingress-nginx-controller-admission   ClusterIP   10.96.193.26   <none>        443/TCP                      22m

创建应用yaml

vim tomcat.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  name: tomcat-deployment

  labels:

    app: tomcat

spec:

  replicas: 2

  selector:

    matchLabels:

      app: tomcat

  minReadySeconds: 1

  progressDeadlineSeconds: 60

  revisionHistoryLimit: 2

  strategy:

    type: RollingUpdate

    rollingUpdate:

      maxSurge: 1

      maxUnavailable: 1

  template:

    metadata:

      labels:

        app: tomcat

    spec:

      containers:

      - name: tomcat

        image: wenlongxue/tomcat:tomcat-demo-62-8fe6052

        imagePullPolicy: Always

        ports:

        - containerPort: 8080

        resources:

          requests:

            memory: "2Gi"

            cpu: "80m"

          limits:

            memory: "2Gi"

            cpu: "80m"

        readinessProbe:

          httpGet:

            path: /

            port: 8080

          initialDelaySeconds: 180

          periodSeconds: 5

          timeoutSeconds: 3

          successThreshold: 1

          failureThreshold: 30

---

apiVersion: v1

kind: Service

metadata:

  name: tomcat-service

  labels:

    app: tomcat

spec:

  selector:

    app: tomcat

  ports:

  - name: tomcat-port

    protocol: TCP

    port: 8080

    targetPort: 8080

  type: ClusterIP

部署 tomcat 应用

kubectl  apply  -f  tomcat.yaml

创建 ingress yaml

vim tomcat-ingress.yaml

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: tomcat

  annotations:

    kubernetes.io/ingress.class: "nginx"

spec:

  rules:

  - host: tomcat.cnsre.cn

    http:

      paths:

      - path: "/"

        pathType: Prefix

        backend:

          service:

            name: tomcat-service

            port:

              number: 8080

部署 tomcat ingress yaml

kubectl  apply  -f  tomcat-ingress.yaml

查看 ingress 对应节点的端口

 kubectl get svc -n ingress-nginx

NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE

ingress-nginx-controller             NodePort    10.96.88.139   <none>        80:30497/TCP,443:32581/TCP   54m

ingress-nginx-controller-admission   ClusterIP   10.96.193.26   <none>        443/TCP                      54m

添加 hosts

在 hosts 文件最后追加 ingress 节点的 IP 地址

54.xxx.xxx.xxx tomcat.cnsre.cn

然后在浏览器中访问 tomcat.cnsre.cn:30497。

使用 hostNetwork 的方式部署 ingress-nginx

每次部署 ingres-nginx 都随机一个 nodePort ，而使用 ingres-nginx 访问的时候也要以 域名:端口 的形式去访问如何直接使用域名去访问呢？下面介绍另外一种安装方式。

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.0/deploy/static/provider/baremetal/deploy.yaml

sed -i 's@k8s.gcr.io/ingress-nginx/controller:v1.0.0\(.*\)@willdockerhub/ingress-nginx-controller:v1.0.0@' deploy.yaml

sed -i 's@k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0\(.*\)$@hzde0128/kube-webhook-certgen:v1.0@' deploy.yaml

优化 ingress-nginx

使用 hostNetwork

默认 ingress-nginx 随机提供 nodeport 端口，开启 hostNetwork 启用80、443端口。

修改 Deployment 下面的 spec

参数如下：

...

    spec:

      hostNetwork: true # 新增

      dnsPolicy: ClusterFirst

      containers:

        - name: controller

          image: willdockerhub/ingress-nginx-controller:v1.0.0  # 更换镜像地址

          imagePullPolicy: IfNotPresent

          lifecycle:

...

修改负载均衡问题

把 kind: Deployment 改为 kind: DaemonSet 模式，这样每台 node 上都有 ingress-nginx-controller pod 副本。

参数如下：

...

# Source: ingress-nginx/templates/controller-deployment.yaml

apiVersion: apps/v1

#kind: Deployment   # 注释

kind: DaemonSet     # 新增

metadata:

  labels:

    helm.sh/chart: ingress-nginx-4.0.1

...

修改 ingressClass 问题

如果不关心 ingressClass 或者很多没有 ingressClass 配置的 ingress 对象，

添加参数 ingress-controller --watch-ingress-without-class=true 。

...

args:

  - /nginx-ingress-controller

  - --publish-service=$(POD_NAMESPACE)/ingress-nginx-dev-v1-test-controller

  - --election-id=ingress-controller-leader

  - --controller-class=k8s.io/ingress-nginx

  - --configmap=$(POD_NAMESPACE)/ingress-nginx-dev-v1-test-controller

  - --validating-webhook=:8443

  - --validating-webhook-certificate=/usr/local/certificates/cert

  - --validating-webhook-key=/usr/local/certificates/key

  - --watch-ingress-without-class=true  # 新增

...

部署检查 ingress

# 部署

kubectl apply -f ingress-nginx.yaml

# 检查 pod

[root@master ~]# kubectl  get  pods -n ingress-nginx  -o wide

NAME                                   READY   STATUS      RESTARTS   AGE   IP               NODE     NOMINATED NODE   READINESS GATES

ingress-nginx-admission-create-gmnmp   0/1     Completed   0          84m   10.100.219.105   master   <none>           <none>

ingress-nginx-admission-patch-f5sgc    0/1     Completed   0          84m   10.100.219.106   master   <none>           <none>

ingress-nginx-controller-b62w7         1/1     Running     0          84m   10.0.10.51       master   <none>           <none>

ingress-nginx-controller-lsn7h         1/1     Running     0          84m   10.0.20.222      node1    <none>           <none>

# 检查端口

[root@master ~]# netstat  -pntl |grep 443

tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      31248/nginx: master

[root@master ~]# netstat  -pntl |grep 80

tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      31248/nginx: master

创建应用yaml

vim tomcat.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  name: tomcat-deployment

  labels:

    app: tomcat

spec:

  replicas: 2

  selector:

    matchLabels:

      app: tomcat

  minReadySeconds: 1

  progressDeadlineSeconds: 60

  revisionHistoryLimit: 2

  strategy:

    type: RollingUpdate

    rollingUpdate:

      maxSurge: 1

      maxUnavailable: 1

  template:

    metadata:

      labels:

        app: tomcat

    spec:

      containers:

      - name: tomcat

        image: wenlongxue/tomcat:tomcat-demo-62-8fe6052

        imagePullPolicy: Always

        ports:

        - containerPort: 8080

        resources:

          requests:

            memory: "2Gi"

            cpu: "80m"

          limits:

            memory: "2Gi"

            cpu: "80m"

        readinessProbe:

          httpGet:

            path: /

            port: 8080

          initialDelaySeconds: 180

          periodSeconds: 5

          timeoutSeconds: 3

          successThreshold: 1

          failureThreshold: 30

---

apiVersion: v1

kind: Service

metadata:

  name: tomcat-service

  labels:

    app: tomcat

spec:

  selector:

    app: tomcat

  ports:

  - name: tomcat-port

    protocol: TCP

    port: 8080

    targetPort: 8080

  type: ClusterIP

部署 tomcat 应用

kubectl  apply  -f  tomcat.yaml

创建 ingress yaml

vim tomcat-ingress.yaml

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: tomcat

  annotations:

    kubernetes.io/ingress.class: "nginx"

spec:

  rules:

  - host: tomcat.cnsre.cn

    http:

      paths:

      - path: "/"

        pathType: Prefix

        backend:

          service:

            name: tomcat-service

            port:

              number: 8080

部署 tomcat ingress yaml

kubectl  apply  -f  tomcat-ingress.yaml

添加 hosts

在 hosts 文件最后追加 ingress 节点的 IP 地址

54.xxx.xxx.xxx tomcat.cnsre.cn

然后在浏览器中访问 tomcat.cnsre.cn:30497。

给 ingress-nginx 配置 HTTPS 访问

创建自签证书文件

openssl req -x509 -nodes -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginx/O=nginx"

创建后会生成两个文件

ll tls.*

-rw-r--r--. 1 root root 1127 9月   2 13:04 tls.crt

-rw-r--r--. 1 root root 1708 9月   2 13:04 tls.key

创建 secret

kubectl create secret tls tls-secret --key tls.key --cert tls.crt

修改 tomcat-ingress yaml

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: tomcat

  annotations:

    kubernetes.io/ingress.class: "nginx"

spec:

  tls:                      # 新增

  - hosts:                  # 新增

    - tomcat.cnsre.cn       # 新增

    secretName: tls-secret  # 新增

  rules:

  - host: tomcat.cnsre.cn

    http:

      paths:

      - path: "/"

        pathType: Prefix

        backend:

          service:

            name: tomcat-service

            port:

              number: 8080

修改完重新部署下

kubectl  apply  -f  tomcat-ingress.yaml

验证证书

访问tomcat.cnsre.cn

文章链接