使用kubeoperator自带的nginx-ingress-controller设置服务的ingress规则进行访问

情况说明

当使用kubeoperator安装k8s集群的时候，在组件设置部分选择的ingress 类型是nginx-ingress

yaml文件

k8s集群安装后，可以在节点的master主机的这个目录下找到使用的yaml文件：/etc/kubernetes/plugins/ingress-controller/nginx-ingress-controller.yaml

该文件内容如下：

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: kube-system
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: kube-system
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: kube-system
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: kube-system
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: kube-system
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: kube-system
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: kube-system
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: kube-system
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        kubernetes.io/os: linux
      hostNetwork: true
      containers:
        - name: nginx-ingress-controller
          image: registry.kubeoperator.io:8082/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          resources:
            limits:
              cpu: 100m
              memory: 128Mi
            requests:
              cpu: 100m
              memory: 128Mi
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
---
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: kube-system
spec:
  type: ClusterIP
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
  - name: https
    port: 443
    targetPort: 443
    protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx

简要分析

通过yaml文件可以查看到，安装的是nginx-ingress-controller，属于kubernetes-ingress-controller, Kubernetes 的“官方”控制器（之所以称为官方，是想把它区别于 NGINX 公司的控制器）。这是社区开发的控制器，它基于 nginx Web 服务器，并补充了一组用于实现额外功能的 Lua 插件。

GitHub地址：https://github.com/kubernetes/ingress-nginx

安装默认使用的namespace是kube-system。(ingress、service、pod、secret 都必须要在同一个 namespace 中，对 ingress-controller 的 namespace 没有要求，因此这里可以使用kube-system，或者新建一个专门的namespace。)

使用示例

这里以一个nginx pod为例，使用Deployment安装俩nginx pod，然后给pod设置service,最后给service设置ingress规则

专门创建一个test命名空间进行操作

kubectl create namespace test

# vim deployment-nginx.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: test
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: c1
        image: nginx:1.15-alpine
        imagePullPolicy: IfNotPresent

# vim nginx-service.yml

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  namespace: test
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: nginx

# vim ingress-nginx.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-nginx        #自定义ingress名称
  namespace: test
spec:
  rules:
  - host: www.daniel.com      # 自定义域名
    http:
      paths:
      - backend:
          serviceName: nginx-service  # 对应上面创建的service名称
          servicePort: 80

测试

在本机(windows10系统)的hosts文件里添加一个域名解析 (前面的ip写k8s集群的任意一个节点ip均可)

192.168.2.163 www.daniel.com

本机浏览器输入域名：www.daniel.com 访问测试

修改其中一个pod nginx的内容进行测试

刷新页面，会发现俩pod对应的nginx显示不同的内容 (因为默认是采用轮询的方式，所以俩不同页面会交替显示)

问题

文件：ingress-nginx.yaml中写的api版本是：apiVersion: extensions/v1beta1,但是实际生成使用的是networking.k8s.io/v1

这是因为在apply这个文件后会有如下的一个提示：Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress

但是直接在文件ingress-nginx.yaml中修改成apiVersion: networking.k8s.io/v1则会报错，这是因为该文件下面的部分内容写法不符合apiVersion: networking.k8s.io/v1规则。

error: error validating "ingress-nginx.yaml": error validating data: [ValidationError(Ingress.spec.rules[0].http.paths[0].backend): unknown field "serviceName" in io.k8s.api.networking.v1.IngressBackend, ValidationError(Ingress.spec.rules[0].http.paths[0].backend): unknown field "servicePort" in io.k8s.api.networking.v1.IngressBackend]; if you choose to ignore these errors, turn validation off with --validate=false

这里还采用之前的写法使用吧，等后续知道该文件下面的部分内容写法符合apiVersion: networking.k8s.io/v1规则后再修改使用apiVersion: networking.k8s.io/v1。

使用kubepoerator自带的kubepi dashboard添加ingress规则

按照页面内容填写需要的信息，提交后会有个报错提示，具体内容还是跟上述说的问题有关

解决办法如下：

不通过页面表单提交了，转而修改页面表单对应的yaml文件，找到apiVersion，把apiVersion: networking.k8s.io/v1beta1修改apiVersion: networking.k8s.io/v1,

此时提交会出现让你填写默认的defaultBackend，没有的话直接删除yaml文件中对应的内容，然后再提交就可以了

在kuboard相对应的页面也会显示的有

kuboard

从上一步可以看出，可以直接使用kubeoperator安装k8s集群时选择的nginx-ingress-controller，就不用在kuboard界面再安装使用了 (再安装相当于是有多个，但是使用的话只能使用一个，多个可以并存的)

官方地址：https://kubernetes.io/zh/docs/concepts/services-networking/ingress-controllers/

点击“手工输入”，内容是：nginx-ingress-controller，然后安装页面内容操作就行了

在KubePi相对应的页面也会显示的有

对上述问题的进一步研究

通过kubepi或kuboard创建好ingress规则后，查看对应的yaml文件，从而可以知道，当使用apiVersion: networking.k8s.io/v1时，ingress-nginx.yaml文件内容写法会有什么变动了,最后引用service这儿有变动

使用extensions/v1beta1

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test-ingress-nginx
  namespace: test
spec:
  rules:
  - host: www.daniel.com
    http:
      paths:
      - backend:
          serviceName: nginx-service
          servicePort: 80

使用networking.k8s.io/v1

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress-nginx
  namespace: test
spec:
  rules:
    - host: www.daniel.com
      http:
        paths:
          - backend:
              service:
                name: nginx-service
                port:
                  number: 80