RHCA cl210 015 实例启动 超融合 热迁移 网络underlay

lab computeresources-hci setup

实例启动流程

keystone不仅做认证，且有所有组键地址

nova-conductor解耦，不允许nova-compute直接访问database

学习方法

学会画逻辑图，且看图比看文字来的要快。文字可能会帮助理解

将学会的东西说出来

超融合架构

存储+计算

nova-compute发放云主机 cpu 内存 网络全使用当前计算节点的资源

实例的磁盘来自单独的ceph集群

超融合既可以跑实例又可以存磁盘数据

红帽提供两种超融合架构

RHV+RHGS

RHOSP13+RHCS3

超融合为分布式 一部分属于ceph 一部分属于openstack

部署角色

(undercloud) [stack@director templates]$ pwd
/home/stack/templates
(undercloud) [stack@director templates]$ ls roles_data.yaml

环境变量 34.定义各个网段的ip

(undercloud) [stack@director classroom-environment]$ ls
00-node-info.yaml                 32-network-environment.yaml
02-custom-domain.yaml             34-ips-from-pool-all.yaml
04-custom-hostnames.yaml          36-fixed-ip-vips.yaml
10-inject-trust-anchor.yaml       40-octavia.yaml
12-low-memory-usage.yaml          42-neutron-ovn-dvr-ha.yaml
14-disable-telemetry.yaml         50-ceph-ansible.yaml
16-telemetry-optimization.yaml    52-ceph-config.yaml
18-compute-extraconfig.yaml       54-ceph-mds.yaml
22-change_root_password-env.yaml  55-manila-cephfsnative-config.yaml
24-admin-password.yaml            90-local_registry_images.yaml
30-network-isolation.yaml         95-idm-integration.yaml
(undercloud) [stack@director classroom-environment]$ pwd
/home/stack/templates/classroom-environment
(undercloud) [stack@director classroom-environment]$

超融合将hypervisor计算角色与同一节点上的本地Ceph OSD相整合

超融合

计算资源和存储资源会互相争夺资源

(计算服务感受不到ceph也在使用资源)

所以需要设置资源约束

工作负载参数

reserved_host_memory 为计算资源预留多少资源

例如 一个节点具有256G内存和10个OSD，每个OSD通常消耗3GB内存，其中分配30GB用于ceph，其余226GB用于计算，这就能托管113个分别使用2GB内存的实例

cpu_allocation_ration参数

默认值16

设置低一点，消耗就会低一点

如果你cpu上限高，这个值就可以设置高一点。更好的压榨cpu

ceph回填

这是需要资源的，osd损坏了的话，会在其他osd恢复，osd会重平衡

超融合实验

用管理员用户添加主机集合



切换普通用户创建实例



选no不需要他创建1个g的卷

创建实例后，直接报错？



not valid host was found

回到管理员



删除主机集合

超融合需要配合flavor

创建flavor并设置metadata

主句聚合有

AZ与Host Aggregates

不输入AZ就不会创建新的AZ

添加超融合节点后更新主机组元数据

完成更新主机聚合的元数据

管理员才能查看hci在哪个主机上

[student@workstation ~(architect1-finance)]$ openstack server list
+--------------------------------------+------+--------+----------+-------+--------+
| ID                                   | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+----------+-------+--------+
| 8cab1c9b-7f03-41b1-81fc-f7980743cc8e | hci  | BUILD  |          |       | hci    |
+--------------------------------------+------+--------+----------+-------+--------+
[student@workstation ~(architect1-finance)]$ openstack server show hci
+-------------------------------------+------------------------------------------------------------------+
| Field                               | Value                                                            |
+-------------------------------------+------------------------------------------------------------------+
| OS-DCF:diskConfig                   | AUTO                                                             |
| OS-EXT-AZ:availability_zone         | nova                                                             |
| OS-EXT-SRV-ATTR:host                | compute1.overcloud.example.com                                   |
| OS-EXT-SRV-ATTR:hypervisor_hostname | compute1.overcloud.example.com

有问题没有达到效果，云主机没有去到超融合节点

命令行再试试

管理员创建主机聚合

[student@workstation ~(architect1-finance)]$ openstack aggregate create hci-aggregate
+-------------------+----------------------------+
| Field             | Value                      |
+-------------------+----------------------------+
| availability_zone | None                       |
| created_at        | 2020-05-31T19:11:50.772175 |
| deleted           | False                      |
| deleted_at        | None                       |
| id                | 5                          |
| name              | hci-aggregate              |
| updated_at        | None                       |
+-------------------+----------------------------+
[student@workstation ~(architect1-finance)]$ openstack aggregate add host hci-aggregate computehci0.overcloud.example.com
+-------------------+----------------------------------------+
| Field             | Value                                  |
+-------------------+----------------------------------------+
| availability_zone | None                                   |
| created_at        | 2020-05-31T19:11:50.000000             |
| deleted           | False                                  |
| deleted_at        | None                                   |
| hosts             | [u'computehci0.overcloud.example.com'] |
| id                | 5                                      |
| metadata          | {}                                     |
| name              | hci-aggregate                          |
| updated_at        | None                                   |
+-------------------+----------------------------------------+
[student@workstation ~(architect1-finance)]$

添加元数据

[student@workstation ~(architect1-finance)]$ openstack aggregate set --property hci=true hci-aggregate
[student@workstation ~(architect1-finance)]$ openstack aggregate show hci-aggregate
+-------------------+----------------------------------------+
| Field             | Value                                  |
+-------------------+----------------------------------------+
| availability_zone | None                                   |
| created_at        | 2020-05-31T19:11:50.000000             |
| deleted           | False                                  |
| deleted_at        | None                                   |
| hosts             | [u'computehci0.overcloud.example.com'] |
| id                | 5                                      |
| name              | hci-aggregate                          |
| properties        | hci='true'                             |
| updated_at        | None                                   |
+-------------------+----------------------------------------+

创建flavor

[student@workstation ~(architect1-finance)]$ openstack flavor create --ram 1024 --disk 10 --vcpus 2 --public default-hci
+----------------------------+--------------------------------------+
| Field                      | Value                                |
+----------------------------+--------------------------------------+
| OS-FLV-DISABLED:disabled   | False                                |
| OS-FLV-EXT-DATA:ephemeral  | 0                                    |
| disk                       | 10                                   |
| id                         | d4b5695e-eca8-4c3e-bd1d-5e8e274ac6db |
| name                       | default-hci                          |
| os-flavor-access:is_public | True                                 |
| properties                 |                                      |
| ram                        | 1024                                 |
| rxtx_factor                | 1.0                                  |
| swap                       |                                      |
| vcpus                      | 2                                    |
+----------------------------+--------------------------------------+

添加元数据

[student@workstation ~(architect1-finance)]$ openstack flavor set --property hci=true default-hci

创建云主机 （也可也用普通用户developer创建）

[student@workstation ~(architect1-finance)]$ openstack server create --flavor default-hci --image hci-image --nic net-id=finance-network1 hci-server1

flavor得是1G内存才能发到超融合里。超融合没能力跑2G内存

主机聚合配合flavor，主机聚合与flavor都得打上一样的标签。flavor创建的虚拟机才会选择相关主机聚合中的主机

lab computeresources-hci setup

清环境

管理计算节点

准备环境

[student@workstation ~(architect1-finance)]$ lab computeresources-migrate setup

创建三个实例

冷迁移需要停业务，底层没有共享文件

热迁移

不中断业务迁移，不会察觉中断

两种方法，一种是块（需要时间迁移块），一种是共享存储vms（只需要迁移内存中的数据）

[heat-admin@controller0 ~]$ rbd -p vms ls
0111edda-900f-420e-8aae-0aaa44e44fae_disk
da04be11-1dbf-44a8-8e63-ca7d819157e5_disk

差分盘在同一个存储

热迁移，底层是要有母盘的，差分盘在共享文件里面，迁的是内存

有一个短暂的时间不接受请求。把那一点内存赶紧迁完

绑定外网ip

[student@workstation ~(architect1-finance)]$ openstack server add floating ip finance-server2  172.25.250.101

迁移完成且业务没有中断

命令行

openstack server migrate 主机 --live compute1.overcloud.example.com

[student@workstation ~(architect1-finance)]$ openstack server list --host compute1.overcloud.example.com --all-projects
+--------------------------------------+-----------------+--------+----------------------------------------------+-------+--------+
| ID                                   | Name            | Status | Networks                                     | Image | Flavor |
+--------------------------------------+-----------------+--------+----------------------------------------------+-------+--------+
| 0111edda-900f-420e-8aae-0aaa44e44fae | finance-server2 | ACTIVE | finance-network1=192.168.1.8, 172.25.250.101 | rhel7 | m1.web |
+--------------------------------------+-----------------+--------+-----------------------------

当节点坏了，可以撤离。撤离后别人不能把业务调度到当前节点上

[student@workstation ~(architect1-finance)]$ openstack compute service list
+----+------------------+-----------------------------------+----------+---------+-------+----------------------------+
| ID | Binary           | Host                              | Zone     | Status  | State | Updated At                 |
+----+------------------+-----------------------------------+----------+---------+-------+----------------------------+
|  1 | nova-conductor   | controller0.overcloud.example.com | internal | enabled | up    | 2020-05-31T17:53:30.000000 |
|  2 | nova-compute     | compute1.overcloud.example.com    | nova     | enabled | up    | 2020-05-31T17:53:27.000000 |
|  3 | nova-compute     | computehci0.overcloud.example.com | nova     | enabled | up    | 2020-05-31T17:53:33.000000 |
|  4 | nova-compute     | compute0.overcloud.example.com    | nova     | enabled | up    | 2020-05-31T17:53:27.000000 |
|  5 | nova-consoleauth | controller0.overcloud.example.com | internal | enabled | up    | 2020-05-31T17:53:29.000000 |
|  6 | nova-scheduler   | controller0.overcloud.example.com | internal | enabled | up    | 2020-05-31T17:53:33.000000 |
+----+------------------+-----------------------------------+----------+---------+-------+----------------------------+

得将节点改为当前不可用，禁用掉

[student@workstation ~(architect1-finance)]$ openstack compute service set compute1.overcloud.example.com  nova-compute --disable
[student@workstation ~(architect1-finance)]$ openstack compute service list
+----+------------------+-----------------------------------+----------+----------+-------+----------------------------+
| ID | Binary           | Host                              | Zone     | Status   | State | Updated At                 |
+----+------------------+-----------------------------------+----------+----------+-------+----------------------------+
|  1 | nova-conductor   | controller0.overcloud.example.com | internal | enabled  | up    | 2020-05-31T17:55:20.000000 |
|  2 | nova-compute     | compute1.overcloud.example.com    | nova     | disabled | up    | 2020-05-31T17:55:20.000000 |
|  3 | nova-compute     | computehci0.overcloud.example.com | nova     | enabled  | up    | 2020-05-31T17:55:23.000000 |
|  4 | nova-compute     | compute0.overcloud.example.com    | nova     | enabled  | up    | 2020-05-31T17:55:17.000000 |
|  5 | nova-consoleauth | controller0.overcloud.example.com | internal | enabled  | up    | 2020-05-31T17:55:19.000000 |
|  6 | nova-scheduler   | controller0.overcloud.example.com | internal | enabled  | up    | 2020-05-31T17:55:23.000000 |
+----+------------------+-----------------------------------+----------+----------+-------+----------------------------+

驱逐节点上所有主机

[student@workstation ~(architect1-finance)]$ nova host-evacuate-live compute1.overcloud.example.com
+--------------------------------------+-------------------------+---------------+
| Server UUID                          | Live Migration Accepted | Error Message |
+--------------------------------------+-------------------------+---------------+
| 0111edda-900f-420e-8aae-0aaa44e44fae | True                    |               |
+--------------------------------------+-------------------------+---------------+

[student@workstation ~(architect1-finance)]$ openstack compute service set compute1.overcloud.example.com nova-compute --enable

neutron

osp网络组键更新较大

SDN实现二层网络虚拟化 （以前ovs，ovs虚拟交换机） NFV实现三层及以上虚拟化

OVN 开放虚拟网络 属于openstack项目

OVN是RHOSP的默认SDN,SDN默认使用OVN

VXLAN Geneve

vlan做隔离 12位vlanid不够用

虚拟机迁移，可能要跨三层。vxlan为大二层，可以使云主机ip地址保持不变

vxlan的 vni隔离可能也不够有

geneve vxlan 都是用vni

geneve 除了增加vni数量，也有更好的适用能力

geneve 元数据可扩展

ovn可以实现防火墙与路由，ovn为ovs的升级 ovn使用geneve协议

vlan20 实例之间的传输

[root@controller0 ~]# vi /var/lib/config-data/neutron/etc/neutron/plugins/ml2/ml2_conf.ini

配置文件

[root@controller0 ~]# lsmod  | grep nf_conntrack
nf_conntrack_netlink    36354  0
nfnetlink              14490  2 nf_conntrack_netlink
nf_conntrack_ipv6      18935  74
nf_defrag_ipv6         35104  2 openvswitch,nf_conntrack_ipv6
nf_conntrack_ipv4      15053  75
nf_defrag_ipv4         12729  1 nf_conntrack_ipv4
nf_conntrack          133053  8 openvswitch,nf_nat,nf_nat_ipv4,nf_nat_ipv6,xt_conntrack,nf_conntrack_netlink,nf_conntrack_ipv4,nf_conntrack_ipv6
libcrc32c              12644  4 xfs,openvswitch,nf_nat,nf_conntrack
[root@controller0 ~]# iptables

ovn桥可以直接调用 模块nf_conntrack
iptables 也用的nf_conntrack

每个节点都有一个分布式路由br-ex osp6只有一个br-ex并放在了网络节点

这样使同一个节点不同网段的实例可以互相访问

也可以使每个节点通过自己的路由去上网，不用去找网络节点

br-int可以直接调用nf_conntrack实现规则，br-int与br-int之间可以直接建立隧道

br-trunk

controller建立了三条隧道

ovs-vsctl-show
Bridge br-int
    fail_mode: secure
    Port "o-hm0"
        Interface "o-hm0"
            type: internal
    Port "ovn-8b62a3-0"
        Interface "ovn-8b62a3-0"
            type: geneve
            options: {csum="true", key=flow, remote_ip="172.24.2.2"}
    Port "patch-br-int-to-provnet-fc5472ee-98d9-4f6b-9bc9-544ca18aefb3"
        Interface "patch-br-int-to-provnet-fc5472ee-98d9-4f6b-9bc9-544ca18aefb3"
            type: patch
            options: {peer="patch-provnet-fc5472ee-98d9-4f6b-9bc9-544ca18aefb3-to-br-int"}
    Port br-int
        Interface br-int
            type: internal
    Port "ovn-028ad0-0"
        Interface "ovn-028ad0-0"
            type: geneve
            options: {csum="true", key=flow, remote_ip="172.24.2.6"}
    Port "ovn-cc9fe3-0"
        Interface "ovn-cc9fe3-0"
            type: geneve
            options: {csum="true", key=flow, remote_ip="172.24.2.12"}

做内网通信的桥，以及安全组桥被合成了一个桥，br-int

各个不同平面的网络，应该在不同平面的网卡上配置ip

eth1被做成br-trunk桥 br-trunk为虚拟交换机vs。eth1为桥的端口

eth1又虚拟出多个子接口，每个子接口为一个ip 172.24.x.x

子接口通过br-trunk出去

eth1既被做成桥也被做成很多个子接口

这是其中一个子接口

[root@controller0 network-scripts]# cat ifcfg-vlan10
# This file is autogenerated by os-net-config
DEVICE=vlan10
ONBOOT=yes
HOTPLUG=no
NM_CONTROLLED=no
PEERDNS=no
DEVICETYPE=ovs
TYPE=OVSIntPort
OVS_BRIDGE=br-trunk
OVS_OPTIONS="tag=10"
BOOTPROTO=static
IPADDR=172.24.1.1
NETMASK=255.255.255.0

子接口通过trunk桥出去

不同网段连在同一个二层设备上。有网络风暴的风险，打上tag隔离广播

vlan20为租户网络

所以实例以后通过vlan20然后trunk出去通信

vlan20和vlan20之间打了隧道

所以第一层壳是172.24这个租户网络 打了tag不会广播，不打会广播 underlay网络

解封装后192，还有vni标签所以不会广播到其他实例

192.168.100.1 192.168.100.2两个不同主机通过vlan20这条隧道ping通

br-trunk可以没有，拿物理交互机可以替代

br-trunk underlay 通过这个出去

租户网络 overlay 解包后得到租户网络

S172.24.2.2

D172.24.2.1

tag隔离风暴

数据包解封装前

S192.168.100.1

D192.168.100.2

VNI隔离

解封装后

实例通过租户网络发数据包，数据包经过隧道（br-int）。重新封装为172租户地址，然后走vlan20，vlan20从trunk出去