MyCat 学习笔记 第十二篇.数据分片 之 分片事务处理
1 环境说明
VM 模拟3台MYSQL 5.6 服务器
VM1 192.168.31.187：3307
VM2 192.168.31.212：3307
VM3 192.168.31.150: 3307
MYCAT 1.5 服务部署在宿主机上
MYCAT 192.168.31.207 :8806【SQL执行端口】 / 9066【管理端口】
2 应用场景
2.0 MYCAT配置
schema.xml
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100">
<table name="t_demo_travel_record" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />
<table name="t_demo_travel_record_child" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />
</schema>
<dataNode name="dn1" dataHost="vm1" database="test" />
<dataNode name="dn2" dataHost="vm2" database="test" />
<dataNode name="dn3" dataHost="vm3" database="test" />
<dataHost name="vm1" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="vm1M1" url="192.168.31.187:3307" user="root" password="root123"></writeHost>
</dataHost>
<dataHost name="vm2" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="vm2M1" url="192.168.31.212:3307" user="root" password="root123"></writeHost>
</dataHost>
<dataHost name="vm3" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="vm3M1" url="192.168.31.150:3307" user="root" password="root123"></writeHost>
</dataHost>
rule.xml
<tableRule name="auto-sharding-long">
<rule>
<columns>id</columns>
<algorithm>rang-long</algorithm>
</rule>
</tableRule>
<function name="rang-long"
class="org.opencloudb.route.function.AutoPartitionByLong">
<property name="mapFile">autopartition-long.txt</property>
<property name="defaultNode">0</property>
</function>
autopartition-long.txt
# range start-end ,data node index
# K=1000,M=10000.
0-500M=0
500M-1000M=1
1000M-1500M=2
2.1 手动关闭事，跨数据库进行更新操作。
用一套简单点的JAVA来做这次数据验证
DataSource ds = jdbc.getDataSource();
conn = ds.getConnection();
conn.setAutoCommit(false);
pstm1 = conn.prepareStatement("insert into t_demo_travel_record (id,context) values (?,?)");
pstm1.setInt(1, 5000);
pstm1.setString(2, "这条记录应该进行DN1");
pstm1.addBatch();
pstm1.setInt(1, 5999999);
pstm1.setString(2, "这条记录应该进行DN2");
pstm1.addBatch();
pstm1.setInt(1, 14991499);
pstm1.setString(2, "这条记录应该进行DN3");
pstm1.addBatch();
pstm1.executeBatch();
conn.commit();
MYCAT 192.168.31.207: 8806 查询已插入的记录
mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);
+----------+-------------+
| id | context |
+----------+-------------+
| 5000 | ????????DN1 |
| 14991499 | ????????DN3 |
| 5999999 | ????????DN2 |
+----------+-------------+
3 rows in set (0.01 sec)
VM1 192.168.31.187：3307 物理库
mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);
+------+-------------+
| id | context |
+------+-------------+
| 5000 | ????????DN1 |
+------+-------------+
1 row in set (0.01 sec)
VM2 192.168.31.212：3307
mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);
+---------+-------------+
| id | context |
+---------+-------------+
| 5999999 | ????????DN2 |
+---------+-------------+
1 row in set (0.00 sec)
VM3 192.168.31.150: 3307
mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);
+----------+-------------+
| id | context |
+----------+-------------+
| 14991499 | ????????DN3 |
+----------+-------------+
1 row in set (0.00 sec)
2.2 异常操作，验证下数据是否回滚
模拟数据如下，最后一条主键重复。
pstm1.setInt(1, 5001);
pstm1.setString(2, "这条记录应该进行DN1");
pstm1.addBatch();
pstm1.setInt(1, 5999998);
pstm1.setString(2, "这条记录应该进行DN2");
pstm1.addBatch();
pstm1.setInt(1, 14991499);
pstm1.setString(2, "这条记录应该进行DN3");
pstm1.addBatch();
程序执行报错：
2016-02-03 00:36:10,229 INFO : [] com.alibaba.druid.pool.DruidDataSource [DruidDataSource.java:669] - {dataSource-1} inited
java.sql.BatchUpdateException: Duplicate entry '14991499' for key 'PRIMARY'
...
再看下数据库里并没有执行前2句SQL语句，mycat 在数据分片插入不同的数据库时已给我们做了一定的数据库事务控制。
mysql> select * from t_demo_travel_record where id in (5001,5999998,14991499);
+----------+-------------+
| id | context |
+----------+-------------+
| 14991499 | ????????DN3 |
+----------+-------------+
1 row in set (0.01 sec)
再来看下MYCAT日志，根据分片原则，分析将SQL语句发向不同的数据结点
02/03 00:36:10.279 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1')
02/03 00:36:10.279 DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1'), route={
1 -> dn1{insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1')}
} rrs
02/03 00:36:10.281 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')
02/03 00:36:10.281 DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2'), route={
1 -> dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}
} rrs
02/03 00:36:10.281 DEBUG [$_NIOREACTOR-1-RW] (MySQLConnection.java:445) -con need syn ,total syn cmd 1 commands SET autocommit=0;schema change:false con:MySQLConnection [id=28, lastTime=1454430970281, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=true, attachment=dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}, respHandler=SingleNodeHandler [node=dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}, packetId=0], host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.284 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')
02/03 00:36:10.284 DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3'), route={
1 -> dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}
} rrs
准备同步数据库操作，发现SQL语句在DN3上编译失败
02/03 00:36:10.285 DEBUG [$_NIOREACTOR-1-RW] (MySQLConnection.java:445) -con need syn ,total syn cmd 1 commands SET autocommit=0;schema change:false con:MySQLConnection [id=8, lastTime=1454430970285, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=true, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, respHandler=SingleNodeHandler [node=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, packetId=0], host=192.168.31.150, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.286 WARN [$_NIOREACTOR-0-RW] (SingleNodeHandler.java:222) -execute sql err : errno:1062 Duplicate entry '14991499' for key 'PRIMARY' con:MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, respHandler=SingleNodeHandler [node=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, packetId=1], host=192.168.31.150, port=3307, statusSync=org.opencloudb.mysql.nio.MySQLConnection$StatusSync@48537945, writeQueue=0, modifiedSQLExecuted=true] frontend host:127.0.0.1/50644/test
再下来开始回滚所有数据结点上的更新操作。
02/03 00:36:10.305 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]rollback
02/03 00:36:10.305 DEBUG [$_NIOREACTOR-1-RW] (RollbackNodeHandler.java:71) -rollback job run for MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, respHandler=SingleNodeHandler [node=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, packetId=1], host=192.168.31.150, port=3307, statusSync=org.opencloudb.mysql.nio.MySQLConnection$StatusSync@48537945, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.305 DEBUG [$_NIOREACTOR-1-RW] (RollbackNodeHandler.java:71) -rollback job run for MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=dn1{insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1')}, respHandler=SingleNodeHandler [node=dn1{insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1')}, packetId=1], host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.306 DEBUG [$_NIOREACTOR-1-RW] (RollbackNodeHandler.java:71) -rollback job run for MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}, respHandler=SingleNodeHandler [node=dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}, packetId=1], host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:361) -clear session resources org.opencloudb.server.NonBlockingSession@603ec61b
释放数据库链接
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, respHandler=org.opencloudb.mysql.nio.handler.RollbackNodeHandler@394047fa, host=192.168.31.150, port=3307, statusSync=org.opencloudb.mysql.nio.MySQLConnection$StatusSync@48537945, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=dn1{insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1')}, respHandler=org.opencloudb.mysql.nio.handler.RollbackNodeHandler@394047fa, host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}, respHandler=org.opencloudb.mysql.nio.handler.RollbackNodeHandler@394047fa, host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (RollbackReleaseHandler.java:58) -autocomit is false,but no commit or rollback ,so mycat rollbacked backend conn MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=org.opencloudb.mysql.nio.handler.RollbackReleaseHandler@2b358b73, host=192.168.31.150, port=3307, statusSync=org.opencloudb.mysql.nio.MySQLConnection$StatusSync@48537945, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=null, host=192.168.31.150, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (RollbackReleaseHandler.java:58) -autocomit is false,but no commit or rollback ,so mycat rollbacked backend conn MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=org.opencloudb.mysql.nio.handler.RollbackReleaseHandler@4856412e, host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=null, host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (RollbackReleaseHandler.java:58) -autocomit is false,but no commit or rollback ,so mycat rollbacked backend conn MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=org.opencloudb.mysql.nio.handler.RollbackReleaseHandler@16d1581c, host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=null, host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
02
OK，通过2个场景验证了MYCAT的事务一致性。
再补一句，大家可以通过MYSQL权威指南文档了解到其实MYCAT实现一种弱XA的事务控制，怎么理解这个弱XA事务控制。
举例有3个结点，MYCAT根据分片规则依次在不同物理库中预执行，如果当中任何一个发现问题则回滚所有。
若预执行成功，3个结点分别进行commit操作时发生异常，则mycat无法回滚已提交的事务。当然，这种机率很小很小~~~
另外一个问题在于，如果发锁升级导致单结点无法操作时，会对整体应用产生比较大的影响。
本篇完
MyCat 学习笔记 第十二篇.数据分片 之 分片事务处理的更多相关文章
- Django学习笔记第十二篇--关于自定义数据库字段数据类型
一.需求背景: django的models模块提供了很多数据字段的数据类型field,但是总有写奇葩需求不能依靠默认字段满足,所以需要自定义数据数据库数据字段类型.所有的自定义field应该在app路 ...
- MyCat 学习笔记 第十篇.数据分片 之 ER分片
1 应用场景 这篇来说下mycat中自带的er关系分片,所谓er关系分片即可以理解为有关联关系表之间数据分片.类似于订单主表与订单详情表间的分片存储规则. 本文所说的er分片分为两种: a. 依据主键 ...
- 汇编入门学习笔记 （十二）—— int指令、port
疯狂的暑假学习之 汇编入门学习笔记 (十二)-- int指令.port 參考: <汇编语言> 王爽 第13.14章 一.int指令 1. int指令引发的中断 int n指令,相当于引 ...
- VSTO 学习笔记（十二）自定义公式与Ribbon
原文:VSTO 学习笔记(十二)自定义公式与Ribbon 这几天工作中在开发一个Excel插件,包含自定义公式,根据条件从数据库中查询结果.这次我们来做一个简单的测试,达到类似的目的. 即在Excel ...
- Binder学习笔记（十二）—— binder_transaction(...)都干了什么？
binder_open(...)都干了什么? 在回答binder_transaction(...)之前,还有一些基础设施要去探究,比如binder_open(...),binder_mmap(...) ...
- Egret入门学习日记 --- 第十二篇（书中 5.1节 内容）
第十二篇(书中 5.1节 内容) 昨天把 第4章完成了. 今天来看第5章. 接下来是 5.1节 的内容. 总结一下 5.1节 的重点: 1.如何制作一个公用按钮皮肤. 跟着做: 重点1:如何制作一个公 ...
- MySQL数据库学习笔记（十二）----开源工具DbUtils的使用（数据库的增删改查）
[声明] 欢迎转载,但请保留文章原始出处→_→ 生命壹号:http://www.cnblogs.com/smyhvae/ 文章来源:http://www.cnblogs.com/smyhvae/p/4 ...
- java之jvm学习笔记六-十二(实践写自己的安全管理器)(jar包的代码认证和签名) (实践对jar包的代码签名) (策略文件)(策略和保护域) (访问控制器) (访问控制器的栈校验机制) (jvm基本结构)
java之jvm学习笔记六(实践写自己的安全管理器) 安全管理器SecurityManager里设计的内容实在是非常的庞大,它的核心方法就是checkPerssiom这个方法里又调用 AccessCo ...
- Android学习笔记（十二）——实战：制作一个聊天界面
//此系列博文是<第一行Android代码>的学习笔记,如有错漏,欢迎指正! 运用简单的布局知识,我们可以来尝试制作一个聊天界面. 一.制作 Nine-Patch 图片 : Nine-Pa ...
随机推荐
- dp --- 2014 Asia AnShan Regional Contest --- HDU 5074 Hatsune Miku
Hatsune Miku Problem's Link: http://acm.hdu.edu.cn/showproblem.php?pid=5074 Mean: 有m种音符(note),现在要从 ...
- C#递归题目代码
一列数的规则如下: 1.1.2.3.5.8.13.21.34...... 求第30位数是多少, 用递归算法实现. 代码: public class MainClass { public static ...
- Winform调用QQ发信息并且开机启动 (开源)
前言 公司CS系统需要加入启动qq从winform调用qq聊天窗口的功能,前提是需要将聊天者的QQ号码作为参数传递到函数中,一直没有搞过,正好很感兴趣,就折腾,Winform调用qq,我想肯定是需要一 ...
- C# Web Forms - Using jQuery FullCalendar
<html xmlns="http://www.w3.org/1999/xhtml"> <head> <title></title> ...
- [js开源组件开发]js多选日期控件
js多选日期控件 详情请见:http://www.lovewebgames.com/jsmodule/calendar.html 它的github地址:https://github.com/tianx ...
- 再谈visibility:hidden和display:none
之前写过一篇有关visibility:hidden和display:none的文章:为什么要用用visibility:hidden;代替display:none;?主要是从浏览器性能方面入手,却没写两 ...
- Matlab2014a 提示未找到支持的编译器或 SDK的解决方法
最近在写论文,用到了matlab版本的libsvm,在混合编译的时候遇到的一点小问题. 我电脑上装的是matlab2014a,window7 64位 >> mbuild -setup 错误 ...
- javascript数组浅谈2
上次说了数组元素的增删,的这次说说数组的一些操作方法 join()方法: ,,] arr.join("_") //1_2_3 join方法会返回一个由数组中每个值的字符串形式拼接而 ...
- ASP.NET MVC another entity of the same type already has the same primary key value
ASP.NET MVC项目 Repository层中,Update.Delete总是失败 another entity of the same type already has the same pr ...
- 转：【前端福利】用grunt搭建自动化的web前端开发环境-完整教程
原文地址:http://blog.csdn.net/wangfupeng1988/article/details/46418203 jQuery在使用grunt,bootstrap在使用grunt,百 ...