KingbaseES R6 集群在线删除standby节点

案例环境：

操作系统：
   [root@node1 ~]# cat /etc/centos-release
CentOS Linux release 7.2.1511 (Core) 
​
数据库：
test=# select version();
                                                       version                                                       
------------------------------------------------------------------------------
 KingbaseES V008R006C003B0010 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46), 64-bit
(1 row)

集群架构：

一、查看当前集群节点状态信息

1、集群节点信息

[kingbase@node3 bin]$ ./repmgr cluster show
 ID | Name    | Role    | Status    | Upstream | Location | Priority | Timeline | Connection string                                                                                                                                
----+---------+---------+-----------+----------+----------+----------+----
 1  | node248 | primary | * running |          | default  | 100      | 5        | host=192.168.7.248 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
 2  | node249 | standby |   running | node248  | default  | 100      | 5        | host=192.168.7.249 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
 3  | node243 | standby |   running | node248  | default  | 100      | 5        | host=192.168.7.243 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3

2、主备流复制信息

test=# select * from sys_stat_replication;
 pid  | usesysid | usename | application_name |  client_addr  | client_hostname | client_port |         backend_start         | backend_xmin |   state   |  sent_lsn  | write_lsn  | flush_lsn  | replay_lsn | write_lag | flush_lag | replay_lag | sync_priority | sync_state |          reply_time    
------+----------+---------+------------------+---------------+-----------
 3403 |    16384 | esrep   | node249          | 192.168.7.249 |                 |       14247 | 2021-03-01 12:03:37.651419+08 |              | streaming | 1/F9005BA0 | 1/F9005BA0 | 1/F9005BA0 | 1/F9005BA0 |           |           |            |             1 | quorum     | 2021-03-01 12:31:56.432910+08
 3419 |    16384 | esrep   | node243          | 192.168.7.243 |                 |       34939 | 2021-03-01 12:03:43.876552+08 |              | streaming | 1/F9005BA0 | 1/F9005BA0 | 1/F9005BA0 | 1/F9005BA0 |           |           |            |             1 | quorum     | 2021-03-01 12:30:42.663663+08
(2 rows)

3、复制槽信息

test=# select * from sys_replication_slots;  
   slot_name   | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn 
---------------+--------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
 repmgr_slot_2 |        | physical  |        |          | f         | t      |       3403 |      |              | 1/F9005BA0  | 
 repmgr_slot_3 |        | physical  |        |          | f         | t      |       3419 |      |              | 1/F9005BA0  | 
(2 rows)

二、将standby节点unregister

1、unregister standby node

在要删除的Standby 节点执行：​
[kingbase@node3 bin]$ ./repmgr standby unregister;
INFO: connecting to local standby
INFO: connecting to primary database
NOTICE: unregistering node 3
INFO: try to drop slot "repmgr_slot_3" of node 3 on primary node
WARNING: replication slot "repmgr_slot_3" is still active on node 3
INFO: standby unregistration complete
​

2、查看集群节点信息

[kingbase@node3 bin]$ ./repmgr cluster show
 ID | Name    | Role    | Status    | Upstream | Location | Priority | Timeline | Connection string                                                                                                                                
----+---------+---------+-----------+----------+----------+----------+----
 1  | node248 | primary | * running |          | default  | 100      | 5        | host=192.168.7.248 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
 2  | node249 | standby |   running | node248  | default  | 100      | 5        | host=192.168.7.249 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3

=== 从以上获知，集群节点信息已经没有unregister的节点信息===

三、停止本地集群服务删除复制槽

1、停止本地集群服务

[kingbase@node3 bin]$ ./sys_monitor.sh stoplocal
Service process "node_export" was killed at process 4928
Service process "postgres_ex" was killed at process 4929
2021-03-01 12:33:52 begin to stop repmgrd on "[localhost]".
2021-03-01 12:33:54 repmgrd on "[localhost]" stop success.
2021-03-01 12:33:54 begin to stop DB on "[localhost]".
waiting for server to shut down.... done
server stopped
2021-03-01 12:33:55 DB on "[localhost]" stop success.

2、关闭cron计划任务：

[kingbase@node2 bin]$ cat /etc/cron.d/KINGBASECRON
#*/1 * * * * kingbase . /etc/profile;/home/kingbase/cluster/R6HA/KHA/kingbase/bin/kbha -A daemon -f /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../etc/repmgr.conf >> /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../kbha.log 2>&1
#*/1 * * * * kingbase  /home/kingbase/cluster/kha/db/bin/network_rewind.sh

3、停止hamgr进程：

[kingbase@node2 bin]$ ps -ef |grep repmgr
kingbase  7536     1  0 12:20 ?        00:01:12 /home/kingbase/cluster/R6HA/KHA/kingbase/bin/repmgrd -d -v -f /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../etc/repmgr.conf
kingbase  7955     1  0 12:21 ?        00:00:01 /home/kingbase/cluster/R6HA/KHA/kingbase/bin/kbha -A daemon -f /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../etc/repmgr.conf

[kingbase@node2 bin]$ kill -2 7536 7955

4、停止export进程：

[kingbase@node2 bin]$ ps -ef |grep kingbase
kingbase  8190     1  0 12:21 ?        00:00:00 /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../share/node_exporter
kingbase  8191     1  0 12:21 ?        00:00:00 /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../share/postgres_exporter
......
[kingbase@node2 bin]$ kill -9 8190 8191 

5、查看复制槽信息

test=# select * from sys_replication_slots;
   slot_name   | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
---------------+--------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
 repmgr_slot_2 |        | physical  |        |          | f         | t      |       3403 |      |              | 1/F9007590  |
 repmgr_slot_3 |        | physical  |        |          | f         | f      |            |      |              | 1/F9007590  |
(2 rows)

6、删除复制槽

test=# select sys_drop_replication_slot('repmgr_slot_3');
 sys_drop_replication_slot
---------------------------

(1 row)

test=# select * from sys_replication_slots;
   slot_name   | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
---------------+--------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
 repmgr_slot_2 |        | physical  |        |          | f         | t      |       3403 |      |              | 1/F9007590  |
(1 row)

7、查看流复制信息

test=# select * from sys_stat_replication;
 pid  | usesysid | usename | application_name |  client_addr  | client_hostname | client_port |         backend_start         | backend_xmin |   state   |  sent_lsn  | write_lsn  | flush_lsn  | replay_lsn | write_lag | flush_lag | replay_lag | sync_priority | sync_state |          reply_time
------+----------+---------+------------------+---------------+-----------------+-------------+-------------------------------+--------------+------
 3403 |    16384 | esrep   | node249          | 192.168.7.249 |                 |       14247 | 2021-03-01 12:03:37.651419+08 |              | streaming | 1/F9007590 | 1/F9007590 | 1/F9007590 | 1/F9007590 |           |           |            |             1 | quorum     | 2021-03-01 12:35:38.819940+08
(1 row)

四、重新启动集群服务测试

[kingbase@node1 bin]$ ./sys_monitor.sh restart
2021-03-01 12:37:43 Ready to stop all DB ...
Service process "node_export" was killed at process 4546
Service process "postgres_ex" was killed at process 4547
Service process "node_export" was killed at process 4242
Service process "postgres_ex" was killed at process 4243
2021-03-01 12:37:49 begin to stop repmgrd on "[192.168.7.248]".
2021-03-01 12:37:50 repmgrd on "[192.168.7.248]" stop success.
2021-03-01 12:37:50 begin to stop repmgrd on "[192.168.7.249]".
2021-03-01 12:37:51 repmgrd on "[192.168.7.249]" stop success.
2021-03-01 12:37:51 begin to stop DB on "[192.168.7.249]".
waiting for server to shut down.... done
server stopped
2021-03-01 12:37:52 DB on "[192.168.7.249]" stop success.
2021-03-01 12:37:52 begin to stop DB on "[192.168.7.248]".
waiting for server to shut down..... done
server stopped
2021-03-01 12:37:53 DB on "[192.168.7.248]" stop success.
2021-03-01 12:37:53 Done.
2021-03-01 12:37:53 Ready to start all DB ...
2021-03-01 12:37:53 begin to start DB on "[192.168.7.248]".
waiting for server to start.... done
server started
2021-03-01 12:37:55 execute to start DB on "[192.168.7.248]" success, connect to check it.
2021-03-01 12:37:56 DB on "[192.168.7.248]" start success.
2021-03-01 12:37:56 Try to ping trusted_servers on host 192.168.7.248 ...
2021-03-01 12:37:59 Try to ping trusted_servers on host 192.168.7.249 ...
2021-03-01 12:38:01 begin to start DB on "[192.168.7.249]".
waiting for server to start.... done
server started
2021-03-01 12:38:03 execute to start DB on "[192.168.7.249]" success, connect to check it.
2021-03-01 12:38:04 DB on "[192.168.7.249]" start success.
 ID | Name    | Role    | Status    | Upstream | Location | Priority | Timeline | Connection string
----+---------+---------+-----------+----------+----------+----------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------
 1  | node248 | primary | * running |          | default  | 100      | 5        | host=192.168.7.248 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
 2  | node249 | standby |   running | node248  | default  | 100      | 5        | host=192.168.7.249 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
2021-03-01 12:38:04 The primary DB is started.
2021-03-01 12:38:08 Success to load virtual ip [192.168.7.240/24] on primary host [192.168.7.248].
2021-03-01 12:38:08 Try to ping vip on host 192.168.7.248 ...
2021-03-01 12:38:11 Try to ping vip on host 192.168.7.249 ...
2021-03-01 12:38:13 begin to start repmgrd on "[192.168.7.248]".
[2021-03-01 12:38:14] [NOTICE] using provided configuration file "/home/kingbase/cluster/R6HA/KHA/kingbase/bin/../etc/repmgr.conf"
[2021-03-01 12:38:14] [NOTICE] redirecting logging output to "/home/kingbase/cluster/R6HA/KHA/kingbase/hamgr.log"

2021-03-01 12:38:14 repmgrd on "[192.168.7.248]" start success.
2021-03-01 12:38:14 begin to start repmgrd on "[192.168.7.249]".
[2021-03-01 12:38:45] [NOTICE] using provided configuration file "/home/kingbase/cluster/R6HA/KHA/kingbase/bin/../etc/repmgr.conf"
[2021-03-01 12:38:45] [NOTICE] redirecting logging output to "/home/kingbase/cluster/R6HA/KHA/kingbase/hamgr.log"

2021-03-01 12:38:15 repmgrd on "[192.168.7.249]" start success.
 ID | Name    | Role    | Status    | Upstream | repmgrd | PID   | Paused? | Upstream last seen
----+---------+---------+-----------+----------+---------+-------+---------+--------------------
 1  | node248 | primary | * running |          | running | 14373 | no      | n/a
 2  | node249 | standby |   running | node248  | running | 12219 | no      | n/a
2021-03-01 12:38:23 Done.

=== 重新启动集群服务，启动后集群节点状态正常，standby节点删除成功===