KingbaseES libstdc++.so.6/ version 'CXXABI_1.3.8'问题处理

ERROR：libstdc++.so.6: version `CXXABI_1.3.8' not found (required by ...)

此文是以 CentOS Linux 7 (AltArch)操作系统为例，编译安装高版本 GCC 编译器的具体过程，此过程也完全适用于其它 Linux 发行版，只是在个别细节上（如使用 yum 还是 apt 或其他软件包管理器），读者需要灵活调整，修改为自己所用操作系统环境支持的指令。

一 系统信息：CentOS Linux release 7.9.2009 (AltArch)

# gcc版本
$ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

# g++版本
$ g++ --version
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

二 问题描述：

KingbaseES数据库软件安装完成。

initdb初始化：error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory。

$ initdb -EUTF8 -moracle -Usystem -D /home/kingbase/data --enable-ci
/home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/kingbase: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory
no data was returned by command ""/home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/kingbase" -V"
initdb: error: The program "kingbase" is needed by initdb but was not found in the
same directory as "/home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/initdb".
Check your installation.

三 排查过程：

1. 使用ldd打印程序或者库文件所依赖的共享库列表

# cd /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/
# 进入bin目录后，使用ldd命令打印程序或者库文件所依赖的共享库列表。
$ ldd kingbase
linux-vdso.so.1 =>  (0x0000ffff97c80000)
libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x0000ffff97c20000)
libssl.so.10 => /usr/lib64/libssl.so.10 (0x0000ffff97b90000)
libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x0000ffff97980000)
librt.so.1 => /usr/lib64/librt.so.1 (0x0000ffff97950000)
libdl.so.2 => /usr/lib64/libdl.so.2 (0x0000ffff97920000)
libm.so.6 => /usr/lib64/libm.so.6 (0x0000ffff97860000)
libicui18n.so.52 => /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/./../lib/libicui18n.so.52 (0x0000ffff975f0000)
libuuid.so.1 => /usr/lib64/libuuid.so.1 (0x0000ffff97410000)
libc.so.6 => /usr/lib64/libc.so.6 (0x0000ffff97280000)
/lib/ld-linux-aarch64.so.1 (0x0000ffff97c90000)
libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000ffff97210000)
libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000ffff97100000)
libcom_err.so.2 => /usr/lib64/libcom_err.so.2 (0x0000ffff970d0000)
libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000ffff97070000)
libz.so.1 => /usr/lib64/libz.so.1 (0x0000ffff97030000)
libgcc_s.so.1 => /usr/lib64/libgcc_s.so.1 (0x0000ffff96fe0000)
libicudata.so.52 => /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/./../lib/../lib/libicudata.so.52 (0x0000ffff95960000)
libstdc++.so.6 => not found
libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x0000ffff95930000)
libkeyutils.so.1 => /usr/lib64/libkeyutils.so.1 (0x0000ffff95910000)
libresolv.so.2 => /usr/lib64/libresolv.so.2 (0x0000ffff958d0000)
libselinux.so.1 => /usr/lib64/libselinux.so.1 (0x0000ffff95880000)
libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x0000ffff95820000)

2. 使用find命令查找libstdc++.so.6文件

# 如果使用find / -name 'libstdc++.so.6'没有查找到文件，使用find / -name 'libstdc++*'进行查找。
# 如果不存在libstdc++.so.6文件，存在libstdc++*文件。类似libstdc++.so.6.0.19/20/21 此类文件
# 使用ln -s libstdc++.so.6.0.19/20/21 libstdc++.so.6 新建新符号连接。
$ find / -name 'libstdc++.so.6'
/usr/lib64/libstdc++.so.6
$ find / -name 'libstdc++*'
/usr/lib64/libstdc++.so.6
/usr/lib64/libstdc++.so.6.0.19

# 然后在使用ldd打印程序或者库文件所依赖的共享库列表
# 如果ldd打印结果 libstdc++.so.6 => not found
# 在当前系统用户环境变量添加 export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
# 添加完成后保存退出 source .bash_profile/.bashrc

$ ldd kingbase
./kingbase: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/./../lib/libicuuc.so.52)
linux-vdso.so.1 =>  (0x0000ffff83ab0000)
libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x0000ffff83a50000)
libssl.so.10 => /usr/lib64/libssl.so.10 (0x0000ffff839c0000)
libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x0000ffff837b0000)
librt.so.1 => /usr/lib64/librt.so.1 (0x0000ffff83780000)
libdl.so.2 => /usr/lib64/libdl.so.2 (0x0000ffff83750000)
libm.so.6 => /usr/lib64/libm.so.6 (0x0000ffff83690000)
libicuuc.so.52 => /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/./../lib/libicuuc.so.52 (0x0000ffff83270000)
libuuid.so.1 => /usr/lib64/libuuid.so.1 (0x0000ffff83240000)
libc.so.6 => /usr/lib64/libc.so.6 (0x0000ffff830b0000)
/lib/ld-linux-aarch64.so.1 (0x0000ffff83ac0000)
libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000ffff83040000)
libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000ffff82f30000)
libcom_err.so.2 => /usr/lib64/libcom_err.so.2 (0x0000ffff82f00000)
libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000ffff82ea0000)
libz.so.1 => /usr/lib64/libz.so.1 (0x0000ffff82e60000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000ffff82d30000)
libgcc_s.so.1 => /usr/lib64/libgcc_s.so.1 (0x0000ffff82cf0000)
libicudata.so.52 => /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/./../lib/../lib/libicudata.so.52 (0x0000ffff81670000)
libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x0000ffff81640000)
libkeyutils.so.1 => /usr/lib64/libkeyutils.so.1 (0x0000ffff81620000)
libresolv.so.2 => /usr/lib64/libresolv.so.2 (0x0000ffff815e0000)
libselinux.so.1 => /usr/lib64/libselinux.so.1 (0x0000ffff81590000)
libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x0000ffff81530000)

3. Ldd依然提示libstdc++.so.6: version `CXXABI_1.3.8' not found (required by ...)

# 使用strings命令查看libstdc++.so.6.0.19库文件CXXABI相关内容
# 通过strings命令发现libstdc++.so.6.0.19库文件缺少CXXABI_1.3.8
# 至此，问题基本确定。程序是在高版本gcc环境编译的，当前系统环境gcc版本低于程序编译环境版本。

$ strings libstdc++.so.6.0.19 | grep 'CXXABI'
CXXABI_1.3
CXXABI_1.3.1
CXXABI_1.3.2
CXXABI_1.3.3
CXXABI_1.3.4
CXXABI_1.3.5
CXXABI_1.3.6
CXXABI_1.3.7
CXXABI_TM_1

strings命令是二进制工具集GNU Binutils的一员，用于打印文件中可打印字符串，文件可以是文本文件（test.c），但一般用于打印二进制目标文件、库或可执行文件中的可打印字符。字符串默认至少是4个或更多可打印字符的任意序列，可使用选项改变字符串最小长度。

四 解决方法：

问题原因：CentOs 7默认gcc版本为4.8.5，程序编译环境gcc版本为5.4.0版本。当前系统gcc版本低于程序编译环境gcc版本导致的问题。

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 

$ g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC)

1. 使用系统提供的开发工具包来管理gcc版本：

优点：可以多版本并存，随时切换，不破坏原有gcc环境。

# 1.安装centos-release-scl
yum install centos-release-scl
# 2.安装devtoolset。
# devtoolset-5-gcc*，安装5.*版本的gcc，devtoolset-[这里输入对应数字就会安装对应的gcc版本{5/6/7/8/9}]-gcc*
yum install devtoolset-5-gcc*
# 临时启用对应的devtoolset，可以一次安装多个版本的devtoolset，用下面的命令切换到对应的版本
# 只对本次会话有效，重启会话后还是会变回原来系统的gcc版本
# 安装的devtoolset默认在/opt/rh目录
# 每个gcc对应目录下面都有enable文件，需要启用某个版本，只需要执行
# source /opt/rh/devtoolset-{5/6/7/8/9}/enable
scl enable devtoolset-5 bash
# 替换旧的gcc，设置LD_LIBRARY_PATH指向新版本gcc安装目录

# https://mirrors.aliyun.com/centos-vault/7.9.2009/sclo/?spm=a2c6h.25603864.0.0.8147d00anRzFUg
# 本次使用阿里云主机，版本对应sclo下无文件。采用手动编译gcc处理。

2 手动下载并编译安装gcc-5.4.0版本

# 在服务器使用wget下载gcc-5.4.0版本gcc
wget https://ftp.gnu.org/gnu/gcc/gcc-5.4.0/gcc-5.4.0.tar.bz2
# 下载完成后使用tar解压
tar jxvf gcc-5.4.0.tar.bz2
# 使用yum安装gcc编译依赖包
yum install gmp-devel mpfr-devel libmpc-devel
# 配置安装gcc，建议除解压目录外单独创建gcc-5.4.0-build目录
$ ls -l
drwxr-xr-x 38 16446 ftp        4096 Jun  3  2016 gcc-5.4.0
drwxr-xr-x 34 root  root       4096 Feb  7 09:38 gcc-5.4.0-build
-rw-r--r--  1 root  root   95661481 Jun  3  2016 gcc-5.4.0.tar.bz2
# 由于gcc编译耗时较久，建议后台运行脚本来进行编译安装
# 默认安装目录为/usr/local/也可以使用--prefix选项修改安装目录
mkdir gcc-5.4.0-build
cd gcc-5.4.0-build
../gcc-5.4.0/configure --enable-languages=c,c++ --disable-multilib
make -j$(nproc) && make install
# 安装完成后，将新安装目录添加到环境变量LD_LIBRARY_PATH以使用新安装的gcc
export LD_LIBRARY_PATH=/usr/local/lib64/:$LD_LIBRARY_PATH
# gcc是向下兼容，版本不需要完全对应，高版本兼容低版本。
# 查看新安装的gcc、g++
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/aarch64-unknown-linux-gnu/5.4.0/lto-wrapper
Target: aarch64-unknown-linux-gnu
Configured with: ../gcc-5.4.0/configure --enable-languages=c,c++ --disable-multilib
Thread model: posix
gcc version 5.4.0 (GCC) 

$ g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/aarch64-unknown-linux-gnu/5.4.0/lto-wrapper
Target: aarch64-unknown-linux-gnu
Configured with: ../gcc-5.4.0/configure --enable-languages=c,c++ --disable-multilib
Thread model: posix
gcc version 5.4.0 (GCC) 

# 至此，gcc升级成功。

3. 验证问题是否解决：

# ldd打印程序或者库文件所依赖的共享库列表
# libstdc++.so.6已经指向正确的gcc

$ ldd kingbase
linux-vdso.so.1 =>  (0x0000ffffa3040000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000ffffa2fd0000)
libssl.so.10 => /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/./../lib/libssl.so.10 (0x0000ffffa2f40000)
libcrypto.so.10 => /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/./../lib/libcrypto.so.10 (0x0000ffffa2d40000)
librt.so.1 => /lib64/librt.so.1 (0x0000ffffa2d10000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000ffffa2ce0000)
libm.so.6 => /lib64/libm.so.6 (0x0000ffffa2c20000)
libicui18n.so.52 => /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/./../lib/libicui18n.so.52 (0x0000ffffa29b0000)
libicuuc.so.52 => /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/./../lib/libicuuc.so.52 (0x0000ffffa2800000)
libuuid.so.1 => /lib64/libuuid.so.1 (0x0000ffffa27d0000)
libc.so.6 => /lib64/libc.so.6 (0x0000ffffa2640000)
/lib/ld-linux-aarch64.so.1 (0x0000ffffa3050000)
libz.so.1 => /lib64/libz.so.1 (0x0000ffffa2600000)
libstdc++.so.6 => /usr/local/lib64/libstdc++.so.6 (0x0000ffffa2460000)
libgcc_s.so.1 => /usr/local/lib64/libgcc_s.so.1 (0x0000ffffa2420000)
libicudata.so.52 => /home/kingbase/KingbaseES/V8/KESRealPro/V008R006C007B0012/Server/bin/./../lib/../lib/libicudata.so.52 (0x0000ffffa0da0000)

# 再次执行initdb初始化数据库
# initdb可以正常初始化

$ initdb -Usystem -D /home/kingbase/KingbaseES/V8/data/ --enable-ci

The files belonging to this database system will be owned by user "kingbase".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

The comparision of strings is case-insensitive.
Data page checksums are disabled.

fixing permissions on existing directory /home/kingbase/KingbaseES/V8/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Shanghai
creating configuration files ... ok
Begin setup encrypt device
initializing the encrypt device ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
create security database ... ok
load security database ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing sys_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    sys_ctl -D /home/kingbase/KingbaseES/V8/data/ -l logfile start

# 启动数据库sys_ctl -D /home/kingbase/KingbaseES/V8/data/ start

$ sys_ctl -D /home/kingbase/KingbaseES/V8/data/ start
waiting for server to start....2023-02-07 09:40:13.519 CST [30002] LOG:  sepapower extension initialized
2023-02-07 09:40:13.527 CST [30002] LOG:  starting KingbaseES V008R006C007B0012 on aarch64-unknown-linux-gnu, compiled by gcc (Ubuntu/Linaro 4.9.1-16kord6) 4.9.1, 64-bit
2023-02-07 09:40:13.528 CST [30002] LOG:  listening on IPv4 address "0.0.0.0", port 54321
2023-02-07 09:40:13.528 CST [30002] LOG:  listening on IPv6 address "::", port 54321
2023-02-07 09:40:13.530 CST [30002] LOG:  listening on Unix socket "/tmp/.s.KINGBASE.54321"
2023-02-07 09:40:13.539 CST [30002] LOG:  redirecting log output to logging collector process
2023-02-07 09:40:13.539 CST [30002] HINT:  Future log output will appear in directory "sys_log".
 done
server started

# ksql登录数据库

[kingbase@iZ2ze4pun4mofkw011ciezZ ~]$ ksql -Usystem -dtest
ksql (V8.0)
Type "help" for help.

test=# select version();
                                                        version
------------------------------------------------------------------------------------------------------------------------
 KingbaseES V008R006C007B0012 on aarch64-unknown-linux-gnu, compiled by gcc (Ubuntu/Linaro 4.9.1-16kord6) 4.9.1, 64-bit
(1 row)

# 问题解决。

五 编译安装gcc一键脚本 ：

#!/bin/bash

echo "Downloading gcc source files..."
wget https://ftp.gnu.org/gnu/gcc/gcc-5.4.0/gcc-5.4.0.tar.bz2 

echo "extracting files..."
tar jxvf gcc-5.4.0.tar.bz2

echo "Installing dependencies..."
yum -y install gmp-devel mpfr-devel libmpc-devel

echo "Configure and install..."
mkdir gcc-5.4.0-build
cd gcc-5.4.0-build
../gcc-5.4.0/configure --enable-languages=c,c++ --disable-multilib
make -j$(nproc) && make install