KingbaseES V8R6 运维系列 --单机小版本升级

​

案例说明：

在KingbaseES V8R6版本提供了sys_upgrade的升级工具， 本案例描述了KingbaseES V8R6单机环境下数据库的小版本升级操作，案例涉及的版本从‘(Kingbase) V008R006C005B0041’通过sys_upgrade升级到‘ (Kingbase) V008R006C005B0054’，此案例可以用于生产环境下数据库升级的参考。

适用版本：

KingbaseES V8R6

升级工具sys_upgade：

sys_upgrade — 升级KingbaseES服务器实例。

sys_upgrade 允许将存储在KingbaseES数据文件中的数据升级到一个更高的KingbaseES主版本，而无需进行主版本升级（例如从 V8R6C4 到 V8R6C5）通常所需的数据转储/重载。

sys_upgrade 支持从 V8R6(20211031版本) 及其后版本升级到当前的KingbaseES主版本，包括快照和beta版本。

sys_upgrade 目前暂不支持 Windows 版本的KingbaseES升级。

详情见官网说明：https://help.kingbase.com.cn/v8/admin/reference/ref-server/pgupgrade.html?highlight=sys_upgrade#id4

一、查看升级前后数据库版本信息

#升级前数据库版本
[kingbase@node101 lib]$ cd /opt/Kingbase/ES/V8R6_041/Server/bin
[kingbase@node101 bin]$ ./ksql -V
ksql (Kingbase) V008R006C005B0041

#升级后数据库版本
[kingbase@node101 mnt]$ cd /opt/Kingbase/ES/V8R6_054/Server/bin
[kingbase@node101 bin]$ ./ksql -V
ksql (Kingbase) V008R006C005B0054

---如果旧版本数据库中有新增插件相关的so 库，而新版本数据库中没有的，需要把相关so 拷贝到新版本数据库lib 目录下。

二、升级前原版本相关参数配置

#升级前版本数据库目录
[kingbase@node101 data]$ pwd
/data/kingbase/v8r6_041/data

#关闭原版本的归档
[kingbase@node101 data]$ cat kingbase.conf |grep archive_mode
archive_mode = off              # enables archiving; off, on, or always

#编辑原版本sys_hba.conf文件
[kingbase@node101 data]$ cp sys_hba.conf sys_hba.conf.old
[kingbase@node101 data]$ sed -i "s/scram-sha-256/trust/g" sys_hba.conf
[kingbase@node101 data]$ cat sys_hba.conf
.......
# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
host    all             all             0.0.0.0/0               trust
# IPv6 local connections:
host    all             all             ::1/128                 trust
host    all             all             ::0/0                   trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     trust
host    replication     all             127.0.0.1/32            trust
host    replication     all             ::1/128                 trust
host    replication     all             192.168.1.0/24           trust
host    all             all             0.0.0.0/0              trust

---如上所示，将sys_hba.conf的认证改为trust，升级期间，便于新旧版本之间访问连接。

三、查看原版本字符集相关信息

Tips：

必须保证升级前后版本的数据库字符集及lc_ctype一致。

test=# show server_encoding;
 server_encoding
-----------------
 UTF8
(1 row)

test=# \l+                                                           List of databases
   Name    | Owner  | Encoding | Collate  |    Ctype    | Access privileges | Size  | Tablespace  |                Description
-----------+--------+----------+----------+-------------+-------------------+-------+-------------+--------------------------------------------
 prod      | system | UTF8     | ci_x_icu | en_US.UTF-8 |                   | 12 MB | sys_default |
 security  | system | UTF8     | ci_x_icu | en_US.UTF-8 |                   | 12 MB | sys_default |
 template0 | system | UTF8     | ci_x_icu | en_US.UTF-8 | =c/system        +| 12 MB | sys_default | unmodifiable empty database
           |        |          |          |             | system=CTc/system |       |             |
 template1 | system | UTF8     | ci_x_icu | en_US.UTF-8 | =c/system        +| 12 MB | sys_default | default template for newdatabases
           |        |          |          |             | system=CTc/system |       |             |
 test      | system | UTF8     | ci_x_icu | en_US.UTF-8 |                   | 12 MB | sys_default | default administrative connection database
(5 rows)

四、安装新版本数据库软件及初始化新的实例

Tips：

• 新版本和旧版本的相关参数：字符集、lc_ctype、数据块大小保证一致。
• 必须使用旧版本数据库initdb时使用的操作系统用户。
• 必须使用和旧版本数据库一样的数据库用户，及-U 参数保持一致。
• block_size 和wal_segsize 必须和旧版本数据库的data 保持一致。
• initdb 初始化的所有参数必须和旧版本数据库initdb 时保持一致。
• 若不一致，在后续的升级兼容性检查也会检测出具体信息。

#在新版本数据库初始化实例
[kingbase@node101 bin]$ ./initdb -U system -W --enable-ci -E utf8 --lc-ctype="en_US.UTF-8" -D /data/kingbase/v8r6_054/data

The files belonging to this database system will be owned by user "kingbase".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default text search configuration will be set to "english".
......
Success. You can now start the database server using:

    ./sys_ctl -D /data/kingbase/v8r6_054/data -l logfile start

#查看新版本数据库信息
[kingbase@node101 bin]$ ./ksql -U system test -p 54323
ksql (V8.0)
Type "help" for help.

test=# \l+
                                                               List of databases
   Name    | Owner  | Encoding | Collate  |    Ctype    | Access privileges | Size  | Tablespace  |                Descriptio
n
-----------+--------+----------+----------+-------------+-------------------+-------+-------------+--------------------------
------------------
 security  | system | UTF8     | ci_x_icu | en_US.UTF-8 |                   | 12 MB | sys_default |
 template0 | system | UTF8     | ci_x_icu | en_US.UTF-8 | =c/system        +| 12 MB | sys_default | unmodifiable empty databa
se
           |        |          |          |             | system=CTc/system |       |             |
 template1 | system | UTF8     | ci_x_icu | en_US.UTF-8 | =c/system        +| 12 MB | sys_default | default template for new
databases
           |        |          |          |             | system=CTc/system |       |             |
 test      | system | UTF8     | ci_x_icu | en_US.UTF-8 |                   | 12 MB | sys_default | default administrative co
nnection database
(4 rows)

五、将原版本相关配置文件拷贝到新版本下

[kingbase@node101 data]$ cp kingbase.conf /data/kingbase/v8r6_054/data/
[kingbase@node101 data]$ cp kingbase.auto.conf /data/kingbase/v8r6_054/data/
[kingbase@node101 data]$ cp sys_hba.conf /data/kingbase/v8r6_054/data/

六、版本升级前兼容性检查

#在新版本数据库bin目录下执行sys_upgrade工具
[kingbase@node101 bin]$ ./sys_upgrade -b /opt/Kingbase/ES/V8R6_041/Server/bin -B /opt/Kingbase/ES/V8R6_054/Server/bin -d /data/kingbase/v8r6_041/data -D /data/kingbase/v8r6_054/data -c -p 54325 -P 54323 -U system
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Checking for presence of required libraries                 ok
Checking database user is the install user                  ok
Checking for prepared transactions                          ok

*Clusters are compatible*
---如上所示，新旧版本之间兼容性检查通过。

七、执行版本升级

Tips：

升级前对原数据库执行物理备份，并停止原版本和新版本数据库服务。

#在新版本数据库bin目录下执行sys_upgrade工具
[kingbase@node101 bin]$ ./sys_upgrade -b /opt/Kingbase/ES/V8R6_041/Server/bin -B /opt/Kingbase/ES/V8R6_054/Server/bin -d /data/kingbase/v8r6_041/data -D /data/kingbase/v8r6_054/data -p 54325 -P 54323 -U system
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Creating dump of global objects                             ok
Creating dump of database schemas
                                                            ok
Checking for presence of required libraries                 ok
Checking database user is the install user                  ok
Checking for prepared transactions                          ok

If sys_upgrade fails after this point, you must re-initdb the
new cluster before continuing.

Performing Upgrade
------------------
Analyzing all rows in the new cluster                       ok
Freezing all rows in the new cluster                        ok
Deleting files from new sys_xact                            ok
Copying old sys_xact to new server                          ok
Setting next transaction ID and epoch for new cluster       ok
Deleting files from new sys_multixact/offsets               ok
Copying old sys_multixact/offsets to new server             ok
Deleting files from new sys_multixact/members               ok
Copying old sys_multixact/members to new server             ok
Setting next multixact ID and offset for new cluster        ok
Resetting WAL archives                                      ok
Setting frozenxid and minmxid counters in new cluster       ok
Restoring global objects in the new cluster                 ok
Restoring database schemas in the new cluster
                                                            ok
Copying user relation files
                                                            ok
Copying wallet files                                        ok
Setting next OID for new cluster                            ok
Sync data directory to disk                                 ok
Creating script to analyze new cluster                      ok
Creating script to delete old cluster                       ok

Upgrade Complete
----------------
Optimizer statistics are not transferred by sys_upgrade so,
once you start the new server, consider running:
    ./analyze_new_cluster.sh

Running this script will delete the old cluster's data files:
    ./delete_old_cluster.sh

八、对升级后的数据库执行统计分析

#在新版本数据库bin目录下执行分析脚本
[kingbase@node101 bin]$ ./analyze_new_cluster.sh
This script will generate minimal optimizer statistics rapidly
so your system is usable, and then gather statistics twice more
with increasing accuracy.  When it is done, your system will
have the default level of optimizer statistics.

If you have used ALTER TABLE to modify the statistics target for
any tables, you might want to remove them and restore them after
running this script because they will delay fast statistics generation.

If you would like default statistics as quickly as possible, cancel
this script and run:
    "/opt/Kingbase/ES/V8R6_054/Server/bin/vacuumdb" -U system --all --analyze-only

vacuumdb: error: could not connect to database template1: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/tmp/.s.KINGBASE.54321"?

Done
---以上错误，需要修改脚本中数据库连接服务端口（默认：54321）

#修改脚本中数据库连接端口
[kingbase@node101 bin]$ cat analyze_new_cluster.sh |grep vacuum
echo '    "/opt/Kingbase/ES/V8R6_054/Server/bin/vacuumdb" -U system --all --analyze-only'
"/opt/Kingbase/ES/V8R6_054/Server/bin/vacuumdb" -U system --all --analyze-in-stages -p 54323

#执行脚本分析
[kingbase@node101 bin]$ ./analyze_new_cluster.sh
This script will generate minimal optimizer statistics rapidly
so your system is usable, and then gather statistics twice more
with increasing accuracy.  When it is done, your system will
have the default level of optimizer statistics.

If you have used ALTER TABLE to modify the statistics target for
any tables, you might want to remove them and restore them after
running this script because they will delay fast statistics generation.

If you would like default statistics as quickly as possible, cancel
this script and run:
    "/opt/Kingbase/ES/V8R6_054/Server/bin/vacuumdb" -U system --all --analyze-only

vacuumdb: processing database "prod": Generating minimal optimizer statistics (1 target)
vacuumdb: processing database "security": Generating minimal optimizer statistics (1 target)
vacuumdb: processing database "template1": Generating minimal optimizer statistics (1 target)
vacuumdb: processing database "test": Generating minimal optimizer statistics (1 target)
vacuumdb: processing database "prod": Generating medium optimizer statistics (10 targets)
vacuumdb: processing database "security": Generating medium optimizer statistics (10 targets)
vacuumdb: processing database "template1": Generating medium optimizer statistics (10 targets)
vacuumdb: processing database "test": Generating medium optimizer statistics (10 targets)
vacuumdb: processing database "prod": Generating default (full) optimizer statistics
vacuumdb: processing database "security": Generating default (full) optimizer statistics
vacuumdb: processing database "template1": Generating default (full) optimizer statistics
vacuumdb: processing database "test": Generating default (full) optimizer statistics

Done
---如上所示，数据库分析完成。

九、查看新版本数据库数据（应该和原版本数据一致）

[kingbase@node101 bin]$ ./ksql -U system test -p 54323
ksql (V8.0)
Type "help" for help.

test=# \l
                             List of databases
   Name    | Owner  | Encoding | Collate  |    Ctype    | Access privileges
-----------+--------+----------+----------+-------------+-------------------
 prod      | system | UTF8     | ci_x_icu | en_US.UTF-8 |
 security  | system | UTF8     | ci_x_icu | en_US.UTF-8 |
 template0 | system | UTF8     | ci_x_icu | en_US.UTF-8 | =c/system        +
           |        |          |          |             | system=CTc/system
 template1 | system | UTF8     | ci_x_icu | en_US.UTF-8 | system=CTc/system+
           |        |          |          |             | =c/system
 test      | system | UTF8     | ci_x_icu | en_US.UTF-8 |
(5 rows)

prod=# select * from t1 limit 10;
 id |   name
----+----------
  1 | 524fe608
  2 | a281abeb
  3 | 12596181
  4 | 8c6964fd
  5 | 53dabd51
  6 | 681f8e17
  7 | 7f4747d7
  8 | 05fb8475
  9 | 53c67d00
 10 | 6b993d5e
(10 rows)

十、删除原版本数据

#在新版本数据库bin目录下执行清理脚本
[kingbase@node101 bin]$ pwd
/opt/Kingbase/ES/V8R6_054/Server/bin
[kingbase@node101 bin]$  ./delete_old_cluster.sh

#原版本data目录下数据已经被删除
[kingbase@node101 data]$ ls -lh
total 0

---如上所示，原版本数据库相关数据文件被删除。

十一、恢复升级版本后的sys_hba.conf

[kingbase@node101 data]$ sed -i "s/trust/scram-sha-256/g" sys_hba.conf

十二、升级中常见故障问题

1、新旧版本数据库lc_ctype不同

2、新旧版本数据库block size不同

以上故障问题，在创建新版本实例前，必须查看旧版本数据库的相关配置，保证新版本创建的实例数据库配置和旧版本一致。