18.flannel的配置

Kubernetes网络通信：
    (1) 容器间通信：同一个Pod内的多个容器间的通信, lo
    (2) Pod通信：Pod IP <--> Pod IP
    (3) Pod与Service通信：PodIP <--> ClusterIP
    (4) Service与集群外部客户端的通信； 

CNI:
    flannel
    calico
    canel
    kube-router
    ... 

    解决方案：
        虚拟网桥
        多路复用：MacVLAN
        硬件交换：SR-IOV
[root@master ~]# cat /etc/cni/net.d/-flannel.conflist
{
  "name": "cbr0",
  "plugins": [
    {
      "type": "flannel",
      "delegate": {
        "hairpinMode": true,
        "isDefaultGateway": true
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    }
  ]
}

    flannel：
        支持多种后端：
            VxLAN
                () vxlan
                () Directrouting
            host-gw: Host Gateway   #不推荐，只能在二层网络中，不支持跨网络，如果有成千上万的Pod，容易产生广播风暴
            UDP:  性能差

        flannel的配置参数：
            Network：flannel使用的CIDR格式的网络地址，用于为Pod配置网络功能；
                10.244.0.0/ ->
                    master: 10.244.0.0/
                    node01: 10.244.1.0/
                    ...
                    node255: 10.244.255.0./ 

                10.0.0.0/
                    10.0.0.0/
                    ...
                    10.255.255.0/ 

            SubnetLen：把Network切分子网供各节点使用时，使用多长的掩码进行切分，默认为24位； 

            SubnetMin：10.244.10.0/ 

            SubnetMax: 10.244.100.0/ 

            Backend：vxlan, host-gw, udp
                vxlan:  

flannel Pod间跨主机通信原理抓包[root@master ~]# ip rdefault via 172.20.0.1 dev ens3 proto dhcp metric

10.244.0.0/ dev cni0 proto kernel scope link src 10.244.0.1
10.244.1.0/ via 10.244.1.0 dev flannel. onlink
10.244.2.0/ via 10.244.2.0 dev flannel. onlink
172.17.0.0/ dev docker0 proto kernel scope link src 172.17.0.1
172.20.0.0/ dev ens3 proto kernel scope link src 172.20.0.91 metric  
从路由可以看出 Pod  cni0 -----> flannel.1 -----> 物理网卡  -----> 对端
[root@master ~]# ip a
: lo: <LOOPBACK,UP,LOWER_UP> mtu  qdisc noqueue state UNKNOWN group default qlen
    link/loopback ::::: brd :::::
    inet 127.0.0.1/ scope host lo
       valid_lft forever preferred_lft forever
: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc pfifo_fast state UP group default qlen
    link/ether ::::b2:ca brd ff:ff:ff:ff:ff:ff
    inet 172.20.0.91/ brd 172.20.255.255 scope global noprefixroute dynamic ens3
       valid_lft 2412sec preferred_lft 2412sec
: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc noqueue state UP group default
    link/ether :::::b4 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/ brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
: flannel.: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc noqueue state UNKNOWN group default
    link/ether :e1::::c1 brd ff:ff:ff:ff:ff:ff
    inet 10.244.0.0/ scope global flannel.
       valid_lft forever preferred_lft forever
: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc noqueue state UP group default qlen
    link/ether 0a::0a:f4:: brd ff:ff:ff:ff:ff:ff
    inet 10.244.0.1/ scope global cni0
       valid_lft forever preferred_lft forever
: veth0f580b07@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc noqueue master cni0 state UP group default
    link/ether a2:::9b:b5:dc brd ff:ff:ff:ff:ff:ff link-netnsid
: vethb8510761@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc noqueue master cni0 state UP group default
    link/ether 1a::6b:::fc brd ff:ff:ff:ff:ff:ff link-netnsid
: vethfc114a8b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc noqueue master cni0 state UP group default
    link/ether fa:ec:f9:ee:: brd ff:ff:ff:ff:ff:ff link-netnsid
: veth023640f@if30028: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc noqueue master docker0 state UP group default
    link/ether d2:8a::e6:8f: brd ff:ff:ff:ff:ff:ff link-netnsid
: br-973161700d44: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc noqueue state UP group default
    link/ether :::2e:ec: brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/ brd 172.18.255.255 scope global br-973161700d44
       valid_lft forever preferred_lft forever
: vethfdee34f@if13226: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc noqueue master docker0 state UP group default
    link/ether 7e:3e:6a:e2::f0 brd ff:ff:ff:ff:ff:ff link-netnsid
: veth49aa01b@if13228: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu  qdisc noqueue master br-973161700d44 state UP group default
    link/ether b2:::6e:4d:da brd ff:ff:ff:ff:ff:ff link-netnsid

cni0 提供Pod网络共享的地址来源
flannel.1 是对数据包进行 vxlan封装
[root@master ~]# brctl show cni0
bridge name    bridge id        STP enabled    interfaces
cni0        .0a580af40001    no        veth0f580b07
                            vethb8510761
                            vethfc114a8b
[root@master ~]# brctl show flannel.
bridge name    bridge id        STP enabled    interfaces
flannel.        can't get info Operation not supported
抓包

15:13:26.796845 IP 172.20.0.93.52954 > 172.20.0.76.otv: OTV, flags [I] (0x08), overlay 0, instance 1
IP 10.244.1.175 > 10.244.2.223: ICMP echo request, id 3072, seq 221, length 64