基于Openstack环境下开启SRIOV

主题思想：

先在系统层面修改配置文件，再去openstack里面修改配置文件

compute node系统层面：

先确认下是否含有ixgbe moudle

lsmod |grep ixgbe

修改/etc/default/grub文件，增加sriov信息，修改后文件如下：

root@compute1-bay5 default]# cat grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet intel_iommu=on ixgbe.max_vfs=16"
GRUB_DISABLE_RECOVERY="true"

重新生成grub文件，NOTE:此步会覆盖原有grub文件，建议保存源文件

root@compute1-bay5 default]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-514.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-70b8529f43024bcf8949e45e7a9140a2
Found initrd image: /boot/initramfs-0-rescue-70b8529f43024bcf8949e45e7a9140a2.img
done

所有compute节点修改完成后重启

compute node openstack层面：

安装sriov，默认是没有安装的，可以使用如下命令搜索：

[root@compute1-bay5 ~]# yum search openstack|grep sriov
openstack-neutron-sriov-nic-agent.noarch : Neutron SR-IOV NIC agent

yum-y install openstack-neutron-sriov-nic-agent.noarch

安装完成后会在/etc/neutron/plugins/ml2/下生成sriov_agent.ini的配置文件，修改增加如下内容：

[sriov_nic]
#
# From neutron.ml2.sriov.agent
#
# Comma-separated list of <physical_network>:<network_device> tuples mapping
# physical network names to the agent's node-specific physical network device
# interfaces of SR-IOV physical function to be used for VLAN networks. All
# physical networks listed in network_vlan_ranges on the server should have
# mappings to appropriate interfaces on each agent. (list value)
physical_device_mappings =physnet1:eno1,physnet4:ens2f1  根据实际情况进行网卡的选择

安装完sriov之后，默认服务是disable的，可以通过命令systemctl list-unit-files|grep sriov查看，通过命令systemctlenable neutron-sriov-nic-agent使服务可用，然后在start下systemctl start neutron-sriov-nic-agent。开启之后要在每个compute node上创建VFs以便支持SRIOV的功能，所以要在nova.conf文件内增加pci白名单：

#pci_passthrough_whitelist =
pci_passthrough_whitelist = [{"devname":"eno1", "physical_network":"physnet1"},{"devname":"ens2f1", "physical_network":"physnet4"}]
这个白名单是根据上文中映射关系来决定的，采用全部添加策略
创建完白名单之后，修改参与sriov功能的网卡，参考结果如下：

[root@compute1-bay5 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno1
TYPE=Ethernet
BOOTPROTO=none 初始为dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eno1
UUID=3ae393ff-318c-4f83-b5a8-2dab3e887cab
DEVICE=eno1
ONBOOT=yes 初始为no

所有compute node弄完之后在control node上重启nova以及neutron。

control node openstack层面：

修改nova的conf文件，增加filters字段如下：
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,CoreFilter,PciPassthroughFilter
修改ml2_conf.ini文件 增加字段如下：
mechanism_drivers =openvswitch,sriovnicswitch

[ml2_sriov]
supported_pci_vonder_devs=8086:10ed
agent_required = True
NOTE:
文档中agent_required的值为false，但在测试时发现如果为true则无法启动SR-IOV
supported_pci_vonder_devs=8086:10ed 中的8086:10ed为因特尔网卡的代码，可以通过如下命令查看
lspci -nn|grep Ethe
文档中建议新建一个ml2配置文件而不是直接修改ml2_conf.ini，如果新建文件，还需要在neutron的启动项中添加相应的文件信息，比较麻烦

查看开启与否的命令如下：
 ip link show|more
[root@compute1-bay5 ~]# ip link show|more
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 38:ea:a7:17:01:5c brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 7 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 8 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 9 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 10 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 11 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 12 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 13 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 14 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 15 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 38:ea:a7:17:01:5d brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 7 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 8 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 9 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 10 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 11 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 12 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 13 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 14 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 15 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
4: enp4s16: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000