OpenShift和F5的集成手册

OpenShift和F5的集成步骤，记录如下，如实际操作中有变更会再度编辑修改。

1.整体架构

使用BIG-IP作为Openshift的Router，能实现以下功能：

• 为Services创建BIG-IP本地流量规则
• 提供HTTP/HTTPS路由
• 为Route资源添加BIG-IP健康检查

本文步骤是通过F5替换OpenShift中的Router,整体的架构如下

2.安装步骤

为BIG-IP设备建立节点

• 在OCP集群中建立一个f5-kctlr-openshift-hostsubnet.yaml文件，内容如下

apiVersion: v1
kind: HostSubnet
metadata:
  name: f5-bigip-
  annotations:
    pod.network.openshift.io/fixed-vnid-host: ""
    pod.network.openshift.io/assign-subnet: "true"
# provide a name for the BIG-IP device's host Node
host: f5-bigip-node-
# Provide an IP address to serve as the BIG-IP VTEP in the OpenShift SDN
hostIP: 172.16.1.28

Host为F5的主机名，hostIP为F5的地址

• 建立Host Subnet

oc create -f f5-kctlr-openshift-hostsubnet.yaml
hostsubnet "f5-bigip-01" created

验证一下

oc get hostsubnet
NAME                  HOST                  HOST IP         SUBNET
f5-big-ip             f5-bigip-node         172.16.1.28     10.129.2.0/

BIG-IP系统设置

需要以管理员或资源管理员身份登录BIG-IP系统

建立VXLAN tunnel

• 建立vxlan profile

create /net tunnels vxlan ose-vxlan flooding-type multipoint

• 建立vxlan tunnel

设置local-address为HostSubnet’s hostip

设置key=0允许设备访问所有的openshift的项目和子网

create /net tunnels tunnel openshift_vxlan key  profile ose-vxlan local-address 172.16.1.28

• 生成一个Self IP在VXLAN中

Self IP的地址范围必须落在集群的子网掩码中，可以通过oc get clusternetwork来查看集群的子网掩码

A self IP address is an IP address on the BIG-IPsystem that you associate with a VLAN, to access hosts in that VLAN. By virtue of its netmask, a self IPaddress represents an address space , that is, a range of IP addresses spanning the hosts in the VLAN, rather than a single host address.

Self IP are used for each device. Each BIG-IP has a self-IP in a VLAN which is the IP defined on the interface.

Floating IP are for a cluster. They are VIPs, and this is the IP shared by your cluster members.

create /net self 10.129.2.3/ allow-service none vlan openshift_vxlan

• 在VXLAN中建立一个浮动的self IP

使用Openshift SDN分配给BIG-IP HostSubnet的子网地址

create /net self 10.129.2.4/ allow-service none traffic-group traffic-group- vlan openshift_vxlan

• 验证BIG IP的对象建立

show /net tunnels tunnel openshift_vxlan
show /net running-config self 10.129.2.3/
show /net running-config self 10.129.2.4/

• 建立一个partition

create auth partition OpenShift

• 在partition下建立一个virtual server

部署BIG-IP Controller

• 建立service account

oc create serviceaccount bigip-ctlr [-n kube-system]
serviceaccount "bigip-ctlr" created

• 建立Cluster role和 Cluster Role binding

# For use in OpenShift clusters
apiVersion: v1
kind: ClusterRole
metadata:
  annotations:
    authorization.openshift.io/system-only: "true"
  name: system:bigip-ctlr
rules:
- apiGroups: ["", "extensions"]
  resources: ["nodes", "services", "endpoints", "namespaces", "ingresses", "routes" ]
  verbs: ["get", "list", "watch"]
- apiGroups: ["", "extensions"]
  resources: ["configmaps", "events", "ingresses/status"]
  verbs: ["get", "list", "watch", "update", "create", "patch" ]
- apiGroups: ["", "extensions"]
  resources: ["secrets"]
  resourceNames: ["<secret-containing-bigip-login>"]
  verbs: ["get", "list", "watch"]

---

apiVersion: v1
kind: ClusterRoleBinding
metadata:
    name: bigip-ctlr-role
userNames:
- system:serviceaccount:kube-system:bigip-ctlr
subjects:
- kind: ServiceAccount
  name: bigip-ctlr
roleRef:
  name: system:bigip-ctlr

oc create -f f5-kctlr-openshift-clusterrole.yaml [-n kube-system]
clusterrole "system:bigip-ctlr" created
clusterrolebinding "bigip-ctlr-role" created

• 建立Deployment
  • --bigip-url 为设备的IP
  • --bigip-partition为之前F5下创建的Partition,Openshift
  • --route-vserver-addr 为F5对外提供服务的IP
  • openshift-sdn-name指向tunnel的名字

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: k8s-bigip-ctlr
spec:
  replicas:
  template:
    metadata:
      name: k8s-bigip-ctlr
      labels:
        app: k8s-bigip-ctlr
    spec:
      # Name of the Service Account bound to a Cluster Role with the required
      # permissions
      serviceAccountName: bigip-ctlr
      containers:
        - name: k8s-bigip-ctlr
          image: "f5networks/k8s-bigip-ctlr"
          env:
            - name: BIGIP_USERNAME
              valueFrom:
                secretKeyRef:
                  # Replace with the name of the Secret containing your login
                  # credentials
                  name: bigip-login
                  key: username
            - name: BIGIP_PASSWORD
              valueFrom:
                secretKeyRef:
                  # Replace with the name of the Secret containing your login
                  # credentials
                  name: bigip-login
                  key: password
          command: ["/app/bin/k8s-bigip-ctlr"]
          args: [
            # See the k8s-bigip-ctlr documentation for information about
            # all config options
            # https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest
            "--bigip-username=$(BIGIP_USERNAME)",
            "--bigip-password=$(BIGIP_PASSWORD)",
            "--bigip-url=10.10.10.10",
            "--bigip-partition=openshift",
            "--pool-member-type=cluster",
            "--openshift-sdn-name=/Common/openshift_vxlan",
            "--manage-routes=true",
            "--route-vserver-addr=1.2.3.4",
            "--route-label="App1"
            ]
      imagePullSecrets:
        - name: f5-docker-images
        - name: bigip-login

oc create -f f5-k8s-bigip-ctlr_openshift-sdn.yaml [-n kube-system]
deployment "k8s-bigip-ctlr" created

• 验证部署成功

oc get pods
NAME                              READY     STATUS    RESTARTS   AGE
k8s-bigip-ctlr--s31l4   /       Running             1m

验证

• 建立route

创建项目，创建应用，然后创建Route

apiVersion: v1
kind: Route
metadata:
  labels:
    name: myService
  name: myService-route-unsecured
  annotations:
    # See the k8s-bigip-ctlr documentation for information about
    # all Route Annotations
    # https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest/#supported-route-annotations
    virtual-server.f5.com/balance: least-connections-node
spec:
  host: mysite.example.com
  path: "/myApp"
  port:
    targetPort:
  to:
    kind: Service
    name: myService

在本地hosts文件中添加mysite.example.com到F5的virtual ip,然后浏览器访问

http://mysite.example.com

• 创建https route

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  annotations:
    virtual-server.f5.com/balance: least-connections-node
  labels:
    app: f5-test
  name: f5-test-
spec:
  host: f5-tes-2t.example.com
  tls:
    insecureEdgeTerminationPolicy: Allow
    termination: edge
  to:
    kind: Service
    name: f5-test

浏览器访问https://f5-test.example.com，即能看到应用页面。

• Openshit上创建Service后，F5会自动创建新的Pool，Pool里的资源即为Service下的Pod
• 请求到达F5后，F5根据请求的域名，找到对应的Pool，请求直接到达Pod。不会请求集群里的Route。

详细参考

https://clouddocs.f5.com/containers/v2/openshift/kctlr-use-bigip-openshift.html