3. ClustrixDB 操作
测试数据库
一、 测试分片，存储信息
测试前：
clxm@p2cn1uclx101m_10.248.100.241 /data]$ clx stat
Cluster Name: cle69e350c2c16d729
Cluster Version: 9.1.4
Cluster Status: OK
Cluster Size: 3 nodes - 8 CPUs per Node
Current Node: ip-10-1-3-151 - nid 3nid | Hostname | Status | IP Address | TPS | Used | Total
----+----------------+--------+-------------+-----+-----------------+--------
1 | ip-10-1-3-88 | OK | 10.1.3.88 | 0 | 145.2M (0.19%) | 74.0G
2 | ip-10-1-3-242 | OK | 10.1.3.242 | 0 | 144.7M (0.19%) | 74.0G
3 | ip-10-1-3-151 | OK | 10.1.3.151 | 0 | 144.8M (0.19%) | 74.0G
----+----------------+--------+-------------+-----+-----------------+--------
0 | 434.7M (0.19%) | 222.0G
测试：(造10000条数据)
create database testdb;
use testdb;
CREATE TABLE userinfo_uuid
(
uuid varchar(36) NOT NULL,
name varchar(64) NOT NULL DEFAULT '',
email varchar(64) NOT NULL DEFAULT '',
password varchar(64) NOT NULL DEFAULT '',
dob date DEFAULT NULL,
address varchar(255) NOT NULL DEFAULT '',
city varchar(64) NOT NULL DEFAULT '',
state_id tinyint unsigned NOT NULL DEFAULT '0',
zip varchar(8) NOT NULL DEFAULT '',
country_id smallint unsigned NOT NULL DEFAULT '0',
gender enum('M','F') NOT NULL DEFAULT 'M',
account_type varchar(32) NOT NULL DEFAULT '',
verified tinyint NOT NULL DEFAULT '0',
allow_mall tinyint unsigned NOT NULL DEFAULT '0',
parrent_account int unsigned NOT NULL DEFAULT '0',
closest_airport varchar(3) NOT NULL DEFAULT '',
PRIMARY KEY(uuid),
UNIQUE KEY email (email),
KEY country_id (country_id),
KEY state_id (state_id),
KEY state_id_2 (state_id,city,address)
)ENGINE=InnoDB;DROP FUNCTION IF EXISTS rand_string;
DELIMITER $$
CREATE FUNCTION rand_string(n INT)
RETURNS VARCHAR(255)
BEGIN
DECLARE chars_str varchar(100) DEFAULT 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789';
DECLARE return_str varchar(255) DEFAULT '';
DECLARE i INT DEFAULT 0;
WHILE i < n DO
SET return_str = concat(return_str,substring(chars_str , FLOOR(1 + RAND()*62 ),1));
SET i = i +1;
END WHILE;
RETURN return_str;
END $$
DELIMITER ;DELIMITER $$
DROP PROCEDURE IF EXISTS `insert_userinfo_uuid`$$
CREATE DEFINER=`root`@`127.0.0.1` PROCEDURE `insert_userinfo_uuid`(IN item INTEGER)
BEGIN
DECLARE counter INT;
SET counter = item;
WHILE counter >= 1 DO
insert into userinfo_uuid (uuid,name,email,password,dob,address,city,state_id,zip,country_id,gender,account_type,verified,allow_mall,parrent_account,closest_airport)
values(uuid(),rand_string(64), rand_string(64), rand_string(64), '2010-10-10', rand_string(255), rand_string(64), ceil(rand() * 100), rand_string(8),
ceil(rand() * 100), 'M', rand_string(32), 0, 0, 0, rand_string(3));
SET counter = counter - 1;
END WHILE;
END$$DELIMITER ;
MySQL [testdb]> call `insert_userinfo_uuid`(10000);
Query OK, 1 row affected (13 min 45.81 sec)
测试后：
clxm@p2cn1uclx101m_10.248.100.241 /opt/clustrix]$ clx stat
Cluster Name: cle69e350c2c16d729
Cluster Version: 9.1.4
Cluster Status: OK
Cluster Size: 3 nodes - 8 CPUs per Node
Current Node: ip-10-1-3-151 - nid 3nid | Hostname | Status | IP Address | TPS | Used | Total
----+----------------+--------+-------------+-----+-----------------+--------
1 | ip-10-1-3-88 | OK | 10.1.3.88 | 0 | 161.9M (0.21%) | 74.0G
2 | ip-10-1-3-242 | OK | 10.1.3.242 | 0 | 161.5M (0.21%) | 74.0G
3 | ip-10-1-3-151 | OK | 10.1.3.151 | 0 | 161.3M (0.21%) | 74.0G
----+----------------+--------+-------------+-----+-----------------+--------
0 | 484.7M (0.21%) | 222.0G
二、在线DDL操作
一千万数据（模拟插数据，验证不会锁表）
MySQL [scloud]> alter table sbtest1 add d varchar(10);
Query OK, 0 rows affected (48.01 sec)
监视ALTER的进度
要查看ALTER进程的状态，请使用此SQL。
sql> select * from system.alter_progress; |
主从同步会出现延迟
MySQL [(none)]> show slave status\G
*************************** 1. row ***************************
Slave_Name: default
Slave_Status: Running
Master_Host: 10.1.1.23
Master_Port: 5306
Master_User: clx_repl
Master_Log_File: clustrix-bin
Slave_Enabled: Enabled
Log_File_Seq: 20
Log_File_Pos: 21568627
Last_Error: no error
Connection_Status: Connected
Relay_Log_Bytes_Read: 0
Relay_Log_Current_Bytes: 43719291
Seconds_Behind_Master: 95
1 row in set (0.00 sec)
ALTER完成后，以高并发性访问或具有N * 10 ^ 7行以上的表的群集性能可能会下降。ClustrixDB存储每个表的统计信息和查询计划，并缓存这些值。ClustrixDB对该信息的定期刷新和刷新可能不会足够快地发生，因此可能会影响性能。为避免这种情况，请 使用以下命令在ALTER完成后立即刷新高速缓存 ：
sql> ALTER TABLE ... shell> clx cmd'mysql system -e“ call pdcache_update（）”'; shell> clx cmd'mysql system -e“ call qpc_flush（）”'; |
请注意，qpc_flush和pdcache_update是基于每个节点完成的。因此，应使用clx cmd实用程序执行这些操作，以确保它们在所有节点上运行。
建议将这些命令作为命令&&命令&&命令在外壳中链接在一起，以避免在ALTER完成和缓存刷新之间出现延迟。
四、在线添加从库
主库: 10.1.1.23:5306
从库: 10.1.3.88:5306
主库开启binlog
MySQL [(none)]> CREATE BINLOG 'clustrix-bin' FORMAT='ROW'
主库创建同步账号
MySQL [(none)]> create user 'clx_repl'@'%' identified by '123123';
MySQL [(none)]> Grant REPLICATION SLAVE ON *.* to 'clx_repl'@'%';
主库模拟创建数据
scloudusr@p2cn1uclx101m_10.248.100.241 ~]$ sysbench --mysql-host=localhost --mysql-port=5306 --mysql-user=root --mysql-password= --mysql-db=scloud --table_size=1000000 oltp_insert prepare
主库备份数据
MySQL [(none)]> backup scloud.* to "sftp://scloudusr:Scloud201!@10.1.3.88:2022/tmp/backup" ;
从库恢复数据
RESTORE * FROM "sftp://scloudusr:Scloud201!@10.1.3.88:2022/tmp/backup" REPLICAS = 1; （快速恢复使用1个副本）
从库备份文件找到主库file和pos信息
cat /tmp/backup/metadata/clustrix-bin.000002:103179247
从库配置同步信息
MySQL [(none)]> stop slave;
MySQL [(none)]> CHANGE MASTER TO MASTER_LOG_FILE = 'clustrix-bin.000002', MASTER_LOG_POS = 103179247, MASTER_HOST = '10.1.1.23', MASTER_USER = 'clx_repl', MASTER_PASSWORD = '123123', MASTER_PORT = 5306;
MySQL [(none)]> start slave;
MySQL [(none)]> show slave status\GMySQL [scloud]> show slave status\G
*************************** 1. row ***************************
Slave_Name: default
Slave_Status: Running
Master_Host: 10.1.1.23
Master_Port: 5306
Master_User: clx_repl
Master_Log_File: clustrix-bin
Slave_Enabled: Enabled
Log_File_Seq: 3
Log_File_Pos: 26232091
Last_Error: no error
Connection_Status: Connected
Relay_Log_Bytes_Read: 0
Relay_Log_Current_Bytes: 12483
Seconds_Behind_Master: 225
1 row in set (0.00 sec)
五、数据库账号迁移
导出命令：clustrix_clone_users -H localhost -u root > user_dump.sql
导入命令：mysql -u root -h localhost < user_dump.sql
clxm@p2cn1uclx101m_10.248.100.241 ~]$ clustrix_clone_users -H localhost -u root > user_dump.sql
clxm@p2cn1uclx101m_10.248.100.241 ~]$ ll
total 4
-rw-r--r-- 1 clxm clxm 946 Nov 26 07:28 user_dump.sql
clxm@p2cn1uclx101m_10.248.100.241 ~]$ more user_dump.sql
--
-- Clustrix Users dumpfile ver: 197:46eb173adf75
-- Host: localhost
--GRANT OSAUTH, REPLICATION CLIENT, SHUTDOWN ON *.* TO 'clustrix'@'localhost';
GRANT ALL PRIVILEGES ON `clustrix_dbi`.* TO 'clustrix'@'localhost';
GRANT ALL PRIVILEGES ON `clustrix_statd`.* TO 'clustrix'@'localhost';
GRANT SELECT ON `information_schema`.* TO 'clustrix'@'localhost';
GRANT SELECT ON `system`.* TO 'clustrix'@'localhost';
GRANT ALL PRIVILEGES ON *.* TO 'clustrix_ui'@'127.0.0.1' WITH GRANT OPTION;
GRANT OSAUTH, SHUTDOWN ON *.* TO 'clxm'@'localhost';
GRANT SELECT ON `system`.* TO 'clxm'@'localhost';
GRANT ALL PRIVILEGES ON *.* TO 'clx_maint'@'127.0.0.1' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON *.* TO 'clx_support'@'127.0.0.1' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON *.* TO 'clx_view_definer'@'127.0.0.1' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON *.* TO 'hogan'@'%';
GRANT ALL PRIVILEGES ON *.* TO 'mysql_slave'@'' WITH GRANT OPTION;
clxm@p2cn1uclx101m_10.248.100.241 ~]$ mysql -u root -h localhost < user_dump.sql
问题： 密码没有导出
六、锁等待，kill 会话操作
模拟锁表
在一个节点执行
MySQL [scloud]> begin; select * from scloud.sbtest1 where id = 4 for update;
Query OK, 0 rows affected (0.00 sec)+----+----------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+------+
| id | k | c | pad | d |
+----+----------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+------+
| 4 | 50201319 | 37002370280-58842166667-00026392672-77506866252-09658311935-56926959306-83464667271-94685475868-28264244556-14550208498 | 63947013338-98809887124-59806726763-79831528812-45582457048 | NULL |
+----+----------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+------+
1 row in set (0.00 sec)
在另一个节点执行（会出现卡住等待锁超时）
MySQL [(none)]> begin; select * from scloud.sbtest1 where id = 4 for update;
Query OK, 0 rows affected (0.00 sec)
找到被锁SQL的session_id然后kill掉
MySQL [system]> select concat('kill ',session_id, ';') from sessions s join lockman l on s.xid = l.waiter;
+---------------------------------+
| concat('kill ',session_id, ';') |
+---------------------------------+
| kill 19458; |
+---------------------------------+
1 row in set (0.00 sec)MySQL [system]> kill 19458;
Query OK, 0 rows affected (0.00 sec)
七、数据库升级
原版本：5.0.45-clustrix-9.1.4
升级后版本；5.0.45-clustrix-9.2
升级用户：clustrix
前提条件： 配置集群中 clustrix 用户免密
**升级操作不能回退，只能升，不能降。提前测试环境测试升级操作。
**主从环境，先升级从库。
1、修改/etc/profile,注释掉后五行 （三个节点都操作）
#df -h
#set -o vi
#stty erase ^H
#set -o vi
#stty erase ^H
修改之后执行 source /etc/profile
2、修改 clustrix用户环境变量，添加 :/opt/clustrix/bin
clustrix@p2cn1uclx101m_10.248.100.241 ~]$ cd
clustrix@p2cn1uclx101m_10.248.100.241 ~]$ cat .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin:/opt/clustrix/bin
export PATH
clustrix@p2cn1uclx101m_10.248.100.241 ~]$ ./clustrix-clxnode-9.2-upgrade.el7.sh install
Extracting New Cluster Management Tools... Done
Configuring Upgrade Authentication... Done
Checking for Node Clock Sync... Done
Verifying Payload MD5 Checksum... Done
Verifying dependencies... Done
Checking for Available Disk Space... Done
Pushing package to cluster... Done
Extracting and Verifying Checksums on Upgrade... Done
Preparing UI for upgrade... Done
Installing Upgrade... Done
Disabling Cluster Alerts... Done
Stopping Database... Done
Switching Symlinks to point at 9_2... Done
Restarting database...2 Done
= = = = = = = = = = = = = = = = = = = = = =
ClustrixDB is now ready for use.
= = = = = = = = = = = = = = = = = = = = = =
Updating Clx UI... Done
Enabling Cluster Alerts... Done
Install completed successfully!
clustrix@p2cn1uclx101m_10.248.100.241 ~]$ mysql -uroot
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 18434
Server version: 5.0.45-clustrix-9.2
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [(none)]>
clustrix@p2cn1uclx101m_10.248.100.241 ~]$ ./clustrix-clxnode-9.2-upgrade.el7.sh install
Extracting New Cluster Management Tools...
Done
Configuring Upgrade Authentication... ERROR
Warning: CLX Command Failed
Fix CLX and try again.
2、/etc/profile 问题
clustrix@p2cn1uclx101m_10.248.100.241 ~]$ ./clustrix-clxnode-9.2-upgrade.el7.sh install
Extracting New Cluster Management Tools... Done
Configuring Upgrade Authentication... Done
stty: standard input: Inappropriate ioctl for device
stty: standard input: Inappropriate ioctl for device
stty: standard input: Inappropriate ioctl for device
stty: standard input: Inappropriate ioctl for device
Checking for Node Clock Sync...stty: standard input: Inappropriate ioctl for device
stty: standard input: Inappropriate ioctl for device
stty: standard input: Inappropriate ioctl for device
stty: standard input: Inappropriate ioctl for device
Warning, non-integer retured by clx cmd 'date +%s': 'FilesystemSizeUsedAvailUse%Mountedon'
Check output of this command manually and fix the problem
八、剔除节点
原集群状态
clxm@p2cn1uclx101m_10.248.100.241 ~]$ clx stat
Cluster Name: cle69e350c2c16d729
Cluster Version: 9.2
Cluster Status: OK
Cluster Size: 3 nodes - 8 CPUs per Node
Current Node: ip-10-1-3-88 - nid 1nid | Hostname | Status | IP Address | TPS | Used | Total
----+----------------+--------+-------------+-----+----------------+--------
1 | ip-10-1-3-88 | OK | 10.1.3.88 | 1 | 9.6G (12.9%) | 73.9G
2 | ip-10-1-3-242 | OK | 10.1.3.242 | 0 | 9.7G (13.1%) | 73.9G
3 | ip-10-1-3-151 | OK | 10.1.3.151 | 0 | 9.5G (12.8%) | 73.9G
----+----------------+--------+-------------+-----+----------------+--------
1 | 28.7G (13.0%) | 221.6G
剔除操作：
MySQL [system]> select * from nodeinfo;
+--------+----------------------------+----------------------------+----------------------------+-----------------------------------------------+------------+-------------------+------------+-----------------+------------+------------+---------+--------------+----------------+-------------------+-------+------------+----------+------+
| nodeid | started | ntptime | up_since | hostname | iface_name | iface_mac_addr | iface_ip | iface_connected | mysql_port | be_ip | be_port | control_port | healthmon_port | pnid | cores | first_pass | be_iface | zone |
+--------+----------------------------+----------------------------+----------------------------+-----------------------------------------------+------------+-------------------+------------+-----------------+------------+------------+---------+--------------+----------------+-------------------+-------+------------+----------+------+
| 2 | 2019-11-26 08:51:25.438532 | 2019-11-26 09:12:17.992518 | 2019-11-26 00:34:51.992518 | ip-10-1-3-242.cn-northwest-1.compute.internal | eth0 | 0a:27:62:3d:4b:78 | 10.1.3.242 | 1 | 5306 | 10.1.3.242 | 24378 | 2048 | 3581 | p94f8dacba3bda6a2 | 8 | 0 | eth0 | 0 |
| 3 | 2019-11-26 08:51:24.795313 | 2019-11-26 09:12:17.186910 | 2019-11-26 00:34:50.186910 | ip-10-1-3-151.cn-northwest-1.compute.internal | eth0 | 0a:36:35:de:83:9c | 10.1.3.151 | 1 | 5306 | 10.1.3.151 | 24378 | 2048 | 3581 | pfe8cd271ed7991fa | 8 | 0 | eth0 | 0 |
| 1 | 2019-11-26 08:51:25.519211 | 2019-11-26 09:12:18.003852 | 2019-11-26 00:34:52.003852 | ip-10-1-3-88.cn-northwest-1.compute.internal | eth0 | 0a:a4:d1:f0:b2:32 | 10.1.3.88 | 1 | 5306 | 10.1.3.88 | 24378 | 2048 | 3581 | p6259ec9897eb8bd9 | 8 | 0 | eth0 | 0 |
+--------+----------------------------+----------------------------+----------------------------+-----------------------------------------------+------------+-------------------+------------+-----------------+------------+------------+---------+--------------+----------------+-------------------+-------+------------+----------+------+
3 rows in set (0.00 sec)MySQL [system]> ALTER CLUSTER drop '3';
Query OK, 0 rows affected (0.01 sec)
剔除后状态
clxm@p2cn1uclx101m_10.248.100.241 ~]$ clx stat
Cluster Name: cle69e350c2c16d729
Cluster Version: 9.2
Cluster Status: OK
Cluster Size: 2 nodes - 8 CPUs per Node
Current Node: ip-10-1-3-88 - nid 1nid | Hostname | Status | IP Address | TPS | Used | Total
----+----------------+--------+-------------+-----+----------------+--------
1 | ip-10-1-3-88 | OK | 10.1.3.88 | 0 | 14.3G (19.3%) | 73.9G
2 | ip-10-1-3-242 | OK | 10.1.3.242 | 18 | 14.4G (19.5%) | 73.9G
----+----------------+--------+-------------+-----+----------------+--------
18 | 28.7G (19.4%) | 147.8G
剔除后做rebalance操作，数据总量不变
3. ClustrixDB 操作的更多相关文章
- 35. ClustrixDB 减少device1大小
ClustrixDB中的device1文件用于所有数据库数据.撤消日志.临时表.binlog和ClustrixDB系统对象.ClustrixDB确保device1文件在集群的所有节点上大小相同.一旦得 ...
- 34. ClustrixDB 降低集群的容量-Flex down
有时,可能需要减少集群的容量: 减少高峰事件后的运营成本 为其他目的分配服务器. 删除故障硬件.(参见删除ALTER CLUSTER以删除永久失败的节点.) 在ClustrixDB中缩小集群的过程很简 ...
- 31. ClustrixDB 分布式架构/查询优化器
ClustrixDB查询优化器有何不同 ClustrixDB查询优化器的核心是能够执行一个具有最大并行性的查询和多个具有最大并发性的并发查询.这是通过分布式查询规划器和编译器以及分布式无共享执行引擎实 ...
- 7. ClustrixDB 集群管理
一. ALTER CLUSTER ADD (Flex Up): 添加节点以增加集群大小 添加节点过程: 步骤1.准备节点 提供节点并在每个节点上安装ClustrixDB,使用相同的版本和安装配置.如 ...
- 29. ClustrixDB 分布式架构/并发控制
介绍 ClustrixDB使用多版本并发控制(MVCC)和2阶段锁(2PL)的组合来支持混合的读写工作负载.在我们的系统中,读取器享受无锁快照隔离,而写入器使用2PL来管理冲突.并发控制的组合意味着读 ...
- 28. ClustrixDB 分布式架构/评估模型
本节描述如何在数据库中计算查询.在ClustrixDB中,我们跨节点切片数据,然后将查询发送到数据.这是数据库的基本原则之一,它允许随着添加更多节点而几乎线性地扩展. 有关如何分布数据的概念,请参阅数 ...
- 27. ClustrixDB 分布式架构/一致性、容错和可用性
一致性 许多分布式数据库都采用最终一致性而不是强一致性来实现可伸缩性.但是,最终的一致性会增加应用程序开发人员的复杂性,他们必须针对可能出现的数据不一致的异常进行开发. ClustrixDB提供了一个 ...
- 26. ClustrixDB 分布式架构/数据分片
数据分片 介绍 共享磁盘vs.无共享 分布式数据库系统可分为两大类数据存储架构:(1)共享磁盘和(2)无共享. Shared Disk Architecture Shared Nothing Arch ...
- 22. ClustrixDB 杀掉恶意会话
ClustrixDB提供了几种机制来识别消耗大量系统资源的查询.这样的查询通常是应用程序索引不良或错误的结果. ClustrixDB支持以下语法来杀死查询: KILL [QUERY | CONNECT ...
随机推荐
- cocos2dx[3.2](10) 新回调函数std::bind
在2.x中处理事件需要用到委托代理(delegate),相信学过2.x的触摸事件的同学,都知道创建和移除的流程十分繁琐. 而在3.x中由于加入了C++11的特性,而对事件的分发机制通过事件分发器Eve ...
- 浅谈 MySQL的预编译
之前的一篇 Mybatis中 #{}和${}的区别 中涉及到通过 SQL预编译和 #{} 传值 的方式防止SQL注入. 由此引发了想了解预编译的想法.那么什么是预编译那? 一.三个阶段: 词法和语义解 ...
- Java中File类的基本用法
File类的基本用法 java.io.File类:代表文件和目录.在开发中,读取文件.生成文件.删除文件.修改文件的属性时经常会用到此类. File类的常用构造方法:public File(Strin ...
- EINT DINT ERTM DRTM EALLOW EDIS ESTOP0的理解
本文参考以下资料整理 https://wenku.baidu.com/view/6b0d6906cf84b9d528ea7a66.html http://pangqicheng123.blog.163 ...
- oracle truncate表 恢复操作
truncate恢复表 1.创建测试用表 conn elan/elan create table haha as select * from dba_users; 2.查询表数据 ) from hah ...
- PTA(Basic Level)1031.查验身份证
一个合法的身份证号码由17位地区.日期编号和顺序编号加1位校验码组成.校验码的计算规则如下: 首先对前17位数字加权求和,权重分配为:{7,9,10,5,8,4,2,1,6,3,7,9,10,5,8, ...
- 【转】mysql索引的探究
转自:https://mp.weixin.qq.com/s/XTu7jERv3A0CIAzlECFnlA 相信很多人对于MySQL的索引都不陌生,索引(Index)是帮助MySQL高效获取数据的数据结 ...
- 声明一个LIst类型的数组
ArrayList[] graphArrayList = new ArrayList[4]; for(int i=0;i<graphArrayList.length;i++){ graphArr ...
- Android快捷键大全
参考来源:https://mp.weixin.qq.com/s/T809p17Wt8XHkbLwcQf9ow 1,Ctrl + J 快捷代码列表 2,Ctrl+Alt+O 这个快捷键可以自动导包或删 ...
- python 压缩文件(解决压缩路径问题)
#压缩文件 def Zip_files(): datapath = filepath # 证据路径 file_newname = datapath + '.zip' # 压缩文件的名字 log.deb ...