KingbaseES V8R6 集群运维系列--sys_monitor.sh stop关闭集群分析

案例说明：

对于KingbaseES V8R6集群关闭整个集群通过执行‘sys_monitor.sh stop’命令完成，本案例解析了在执行‘sys_monitor.sh stop’后，数据库的关闭方式及数据库访问带来的影响。

KingbaseES停止数据库方式：（sys_ctl）

参数-m控制数据库停止模式 smart fast immediate：
smart 模式会等待活动的事务提交结束，并等待客户端主动断开连接之后关闭数据库
fast 模式则会回滚所有活动的事务，并强制断客户端的连接之后关闭数据库
immediate 模式立即终止所有服务器进程，当下一次数据库启动时它会首先进入恢复状态，一般不推荐使用。

适用版本：

KingbaseES V8R6

一、集群关闭案例测试

1、查看关闭数据库前控制文件

[kingbase@node101 bin]$ ./sys_controldata -D /data/kingbase/r6ha/data
sys_control version number:            1201
Catalog version number:               202112261
Database system identifier:           7080367334319169673
Database cluster state:               in production
sys_control last modified:             Fri 23 Dec 2022 11:01:58 AM CST
Latest checkpoint location:           4/F5000028
Latest checkpoint's REDO location:    4/F5000028
Latest checkpoint's REDO WAL file:    0000002300000004000000F5
Latest checkpoint's TimeLineID:       35
Latest checkpoint's PrevTimeLineID:   35
......

2、sys_monitor.sh关闭集群

[kingbase@node101 bin]$ ./sys_monitor.sh stop
.......
2022-12-23 11:04:53 begin to stop DB on "[192.168.1.102]".
waiting for server to shut down.... done
server stopped
2022-12-23 11:04:55 DB on "[192.168.1.102]" stop success.
2022-12-23 11:04:55 begin to stop DB on "[192.168.1.101]".
waiting for server to shut down............ done
server stopped
2022-12-23 11:05:06 DB on "[192.168.1.101]" stop success.
2022-12-23 11:05:06 Done.

3、查看主备库sys_log日志

#主库
2022-12-23 11:04:57.625 CST,,,10977,,63a51a26.2ae1,3,,2022-12-23 11:01:58 CST,,0,LOG,00000,"received fast shutdown request",,,,,,,,,""
2022-12-23 11:04:57.638 CST,,,10977,,63a51a26.2ae1,4,,2022-12-23 11:01:58 CST,,0,LOG,00000,"aborting any active transactions",,,,,,,,,""
2022-12-23 11:04:57.643 CST,,,10977,,63a51a26.2ae1,5,,2022-12-23 11:01:58 CST,,0,LOG,00000,"background worker ""kwr collector"" (PID 10988) exited with exit code 1",,,,,,,,,""
2022-12-23 11:04:57.643 CST,,,10977,,63a51a26.2ae1,6,,2022-12-23 11:01:58 CST,,0,LOG,00000,"background worker ""logical replication launcher"" (PID 10989) exited with exit code 1",,,,,,,,,""
2022-12-23 11:04:57.643 CST,"system","prod",12318,"192.168.1.102:58496",63a51a8e.301e,1,"INSERT",2022-12-23 11:03:42 CST,10/118,3274,FATAL,57P01,"terminating connection due to administrator command",,,,,,"insert into t1 select * from t1;",,,"kingbase_*&+_"
2022-12-23 11:04:58.876 CST,,,10980,,63a51a26.2ae4,2,,2022-12-23 11:01:58 CST,,0,LOG,00000,"checkpoint complete: wrote 15484 buffers (94.5%); 1 WAL file(s) added, 0 removed, 0 recycled; write=12.169 s, sync=0.426 s, total=13.518 s; sync files=6, longest=0.224 s, average=0.071 s; distance=694028 kB, estimate=694028 kB",,,,,,,,,""
2022-12-23 11:04:58.876 CST,,,10980,,63a51a26.2ae4,3,,2022-12-23 11:01:58 CST,,0,LOG,00000,"shutting down",,,,,,,,,""
2022-12-23 11:04:58.913 CST,,,10980,,63a51a26.2ae4,4,,2022-12-23 11:01:58 CST,,0,LOG,00000,"checkpoint starting: shutdown immediate",,,,,,,,,""
2022-12-23 11:04:59.494 CST,,,10980,,63a51a26.2ae4,5,,2022-12-23 11:01:58 CST,,0,LOG,00000,"checkpoint complete: wrote 4108 buffers (25.1%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.130 s, sync=0.301 s, total=0.615 s; sync files=2, longest=0.271 s, average=0.150 s; distance=223475 kB, estimate=646973 kB",,,,,,,,,""
2022-12-23 11:05:06.494 CST,,,10977,,63a51a26.2ae1,7,,2022-12-23 11:01:58 CST,,0,LOG,00000,"database system is shut down",,,,,,,,,""

#备库
2022-12-22 17:18:09.565 CST,,,1887,,63a4206f.75f,3,,2022-12-22 17:16:31 CST,,0,LOG,00000,"received fast shutdown request",,,,,,,,,""
2022-12-22 17:18:09.587 CST,,,1887,,63a4206f.75f,4,,2022-12-22 17:16:31 CST,,0,LOG,00000,"aborting any active transactions",,,,,,,,,""
2022-12-22 17:18:09.598 CST,,,1909,,63a42079.775,2,,2022-12-22 17:16:41 CST,,0,FATAL,57P01,"terminating walreceiver process due to administrator command",,,,,,,,,""
2022-12-22 17:18:09.599 CST,,,1906,,63a4206f.772,1,,2022-12-22 17:16:31 CST,,0,LOG,00000,"shutting down",,,,,,,,,""
2022-12-22 17:18:09.599 CST,,,1906,,63a4206f.772,2,,2022-12-22 17:16:31 CST,,0,LOG,00000,"restartpoint starting: shutdown immediate",,,,,,,,,""
2022-12-22 17:18:11.179 CST,,,1906,,63a4206f.772,3,,2022-12-22 17:16:31 CST,,0,LOG,00000,"restartpoint complete: wrote 44628 buffers (68.1%); 0 WAL file(s) added, 0 removed, 35 recycled; write=1.014 s, sync=0.458 s, total=1.580 s; sync files=6, longest=0.174 s, average=0.076 s; distance=573439 kB, estimate=573439 kB",,,,,,,,,""
2022-12-22 17:18:11.179 CST,,,1906,,63a4206f.772,4,,2022-12-22 17:16:31 CST,,0,LOG,00000,"recovery restart point at 1/CD000028","Last completed transaction was at log time 2022-12-22 17:17:06.024532+08.",,,,,,,,""
2022-12-22 17:18:11.241 CST,,,1887,,63a4206f.75f,5,,2022-12-22 17:16:31 CST,,0,LOG,00000,"database system is shut down",,,,,,,,,""
.......

如下图所示：

1）数据库收到“received fast shutdown request"的关库的请求。
2）数据库终止并回滚正在执行的事务，断开客户端的连接。
3）在关闭数据库之前执行了checkpoint。

4、客户端连接测试

#客户端连接数据库访问
[kingbase@node102 bin]$ ./ksql -h 192.168.1.101 -U system test
ksql (V8.0)
Type "help" for help.

test=# \c prod
You are now connected to database "prod" as user "system".

#查看t1表关库前数据
prod=# select count(*) from  t1;
  count
----------
 16000000
(1 row)

#执行insert操作，在数据库关闭时，事务被回滚并且客户端连接被断开
prod=# insert into t1 select * from t1;

FATAL:  terminating connection due to administrator command
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.

5、查看数据库服务关闭后控制文件

[kingbase@node101 bin]$ ./sys_controldata -D /data/kingbase/r6ha/data
sys_control version number:            1201
Catalog version number:               202112261
Database system identifier:           7080367334319169673
Database cluster state:               shut down
sys_control last modified:             Fri 23 Dec 2022 11:04:59 AM CST
Latest checkpoint location:           5/2D000028
Latest checkpoint's REDO location:    5/2D000028
Latest checkpoint's REDO WAL file:    00000023000000050000002D
Latest checkpoint's TimeLineID:       35
Latest checkpoint's PrevTimeLineID:   35
.......

---如上所示，在数据库被关闭后，产生了新的检查点。

6、数据库服务启动后，查询数据（事务被回滚）

[kingbase@node102 bin]$ ./ksql -h 192.168.1.101 -U system test
ksql (V8.0)
Type "help" for help.

test=# \c prod
You are now connected to database "prod" as user "system".
prod=# select count(*) from  t1;
  count
----------
 16000000
(1 row)

二、sys_ctl fast stop关库测试

1、查看关库前检查点

sys_control version number:            1201
Catalog version number:               202202151
Database system identifier:           7100823070678531859
Database cluster state:               in production
sys_control last modified:             Thu 22 Dec 2022 05:30:20 PM CST
Latest checkpoint location:           0/242CFEF0
Latest checkpoint's REDO location:    0/242CFEF0
Latest checkpoint's REDO WAL file:    000000010000000000000024
Latest checkpoint's TimeLineID:       1
Latest checkpoint's PrevTimeLineID:   1
.......

2、sys_ctl fast stop 关闭数据库

[kingbase@node2 bin]$ ./sys_ctl -m fast stop -D /data/kingbase/v8r6_054/data
waiting for server to shut down.... done
server stopped

3、查看sys_log日志

[kingbase@node2 sys_log]$ tail -f kingbase-2022-12-22_165801.log
2022-12-22 16:58:01.351 CST [31158] LOG:  database system was shut down at 2022-12-22 16:56:55 CST
2022-12-22 16:58:01.361 CST [31156] LOG:  database system is ready to accept connections
2022-12-22 16:59:16.314 CST [31156] LOG:  received fast shutdown request
2022-12-22 16:59:16.320 CST [31156] LOG:  aborting any active transactions
2022-12-22 16:59:16.320 CST [31876] FATAL:  terminating connection due to administrator command
2022-12-22 16:59:16.329 CST [31156] LOG:  background worker "kwr collector" (PID 31166) exited with exit code 1
2022-12-22 16:59:16.329 CST [31156] LOG:  background worker "logical replication launcher" (PID 31167) exited with exit code 1
2022-12-22 16:59:16.329 CST [31159] LOG:  shutting down
2022-12-22 16:59:16.534 CST [31156] LOG:  database system is shut down

如下图所示：

1）sys_ctl fast stop将中断事务的操作（回滚 rollback）。
2）终止客户端对数据库的连接访问。
3) shutdown数据库前没有触发checkpoint。

4、查看关库后检查点信息

[kingbase@node2 bin]$ ./sys_controldata  -D /data/kingbase/v8r6_054/data/
sys_control version number:            1201
Catalog version number:               202202151
Database system identifier:           7100823070678531859
Database cluster state:               shut down
sys_control last modified:             Thu 22 Dec 2022 05:32:03 PM CST
Latest checkpoint location:           0/242E5298
Latest checkpoint's REDO location:    0/242E5298
Latest checkpoint's REDO WAL file:    000000010000000000000024
Latest checkpoint's TimeLineID:       1
Latest checkpoint's PrevTimeLineID:   1
........

--如上所示，在执行sys_ctl fast stop关闭数据库后，检查点没有发生变化，在sys_log日志
中没有记录触发检查点的日志信息。

如下图所示：不指定任何参数的sys_ctl关闭数据库（等同于‘fast stop’）

三、总结

在集群执行sys_monitor.sh stop和sys_ctl fast stop关闭数据库服务时：

1）都会调用’fast shutdown’方式关闭数据库。

2）数据库终止并回滚正在执行的事务，断开客户端的连接。

3）sys_monitor.sh stop执行时，数据库shutdown之前将会执行checkpoint，而sys_ctl fast stop不会执行checkpoint。