MyCat 学习笔记 第十二篇.数据分片 之 分片事务处理
1 环境说明
VM 模拟3台MYSQL 5.6 服务器
VM1 192.168.31.187：3307
VM2 192.168.31.212：3307
VM3 192.168.31.150: 3307
MYCAT 1.5 服务部署在宿主机上
MYCAT 192.168.31.207 :8806【SQL执行端口】 / 9066【管理端口】
2 应用场景
2.0 MYCAT配置
schema.xml
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100">
<table name="t_demo_travel_record" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />
<table name="t_demo_travel_record_child" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />
</schema>
<dataNode name="dn1" dataHost="vm1" database="test" />
<dataNode name="dn2" dataHost="vm2" database="test" />
<dataNode name="dn3" dataHost="vm3" database="test" />
<dataHost name="vm1" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="vm1M1" url="192.168.31.187:3307" user="root" password="root123"></writeHost>
</dataHost>
<dataHost name="vm2" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="vm2M1" url="192.168.31.212:3307" user="root" password="root123"></writeHost>
</dataHost>
<dataHost name="vm3" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="vm3M1" url="192.168.31.150:3307" user="root" password="root123"></writeHost>
</dataHost>
rule.xml
<tableRule name="auto-sharding-long">
<rule>
<columns>id</columns>
<algorithm>rang-long</algorithm>
</rule>
</tableRule>
<function name="rang-long"
class="org.opencloudb.route.function.AutoPartitionByLong">
<property name="mapFile">autopartition-long.txt</property>
<property name="defaultNode">0</property>
</function>
autopartition-long.txt
# range start-end ,data node index
# K=1000,M=10000.
0-500M=0
500M-1000M=1
1000M-1500M=2
2.1 手动关闭事，跨数据库进行更新操作。
用一套简单点的JAVA来做这次数据验证
DataSource ds = jdbc.getDataSource();
conn = ds.getConnection();
conn.setAutoCommit(false);
pstm1 = conn.prepareStatement("insert into t_demo_travel_record (id,context) values (?,?)");
pstm1.setInt(1, 5000);
pstm1.setString(2, "这条记录应该进行DN1");
pstm1.addBatch();
pstm1.setInt(1, 5999999);
pstm1.setString(2, "这条记录应该进行DN2");
pstm1.addBatch();
pstm1.setInt(1, 14991499);
pstm1.setString(2, "这条记录应该进行DN3");
pstm1.addBatch();
pstm1.executeBatch();
conn.commit();
MYCAT 192.168.31.207: 8806 查询已插入的记录
mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);
+----------+-------------+
| id | context |
+----------+-------------+
| 5000 | ????????DN1 |
| 14991499 | ????????DN3 |
| 5999999 | ????????DN2 |
+----------+-------------+
3 rows in set (0.01 sec)
VM1 192.168.31.187：3307 物理库
mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);
+------+-------------+
| id | context |
+------+-------------+
| 5000 | ????????DN1 |
+------+-------------+
1 row in set (0.01 sec)
VM2 192.168.31.212：3307
mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);
+---------+-------------+
| id | context |
+---------+-------------+
| 5999999 | ????????DN2 |
+---------+-------------+
1 row in set (0.00 sec)
VM3 192.168.31.150: 3307
mysql> select * from t_demo_travel_record where id in (5000,5999999,14991499);
+----------+-------------+
| id | context |
+----------+-------------+
| 14991499 | ????????DN3 |
+----------+-------------+
1 row in set (0.00 sec)
2.2 异常操作，验证下数据是否回滚
模拟数据如下，最后一条主键重复。
pstm1.setInt(1, 5001);
pstm1.setString(2, "这条记录应该进行DN1");
pstm1.addBatch();
pstm1.setInt(1, 5999998);
pstm1.setString(2, "这条记录应该进行DN2");
pstm1.addBatch();
pstm1.setInt(1, 14991499);
pstm1.setString(2, "这条记录应该进行DN3");
pstm1.addBatch();
程序执行报错：
2016-02-03 00:36:10,229 INFO : [] com.alibaba.druid.pool.DruidDataSource [DruidDataSource.java:669] - {dataSource-1} inited
java.sql.BatchUpdateException: Duplicate entry '14991499' for key 'PRIMARY'
...
再看下数据库里并没有执行前2句SQL语句，mycat 在数据分片插入不同的数据库时已给我们做了一定的数据库事务控制。
mysql> select * from t_demo_travel_record where id in (5001,5999998,14991499);
+----------+-------------+
| id | context |
+----------+-------------+
| 14991499 | ????????DN3 |
+----------+-------------+
1 row in set (0.01 sec)
再来看下MYCAT日志，根据分片原则，分析将SQL语句发向不同的数据结点
02/03 00:36:10.279 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1')
02/03 00:36:10.279 DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1'), route={
1 -> dn1{insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1')}
} rrs
02/03 00:36:10.281 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')
02/03 00:36:10.281 DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2'), route={
1 -> dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}
} rrs
02/03 00:36:10.281 DEBUG [$_NIOREACTOR-1-RW] (MySQLConnection.java:445) -con need syn ,total syn cmd 1 commands SET autocommit=0;schema change:false con:MySQLConnection [id=28, lastTime=1454430970281, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=true, attachment=dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}, respHandler=SingleNodeHandler [node=dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}, packetId=0], host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.284 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')
02/03 00:36:10.284 DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3'), route={
1 -> dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}
} rrs
准备同步数据库操作，发现SQL语句在DN3上编译失败
02/03 00:36:10.285 DEBUG [$_NIOREACTOR-1-RW] (MySQLConnection.java:445) -con need syn ,total syn cmd 1 commands SET autocommit=0;schema change:false con:MySQLConnection [id=8, lastTime=1454430970285, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=true, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, respHandler=SingleNodeHandler [node=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, packetId=0], host=192.168.31.150, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.286 WARN [$_NIOREACTOR-0-RW] (SingleNodeHandler.java:222) -execute sql err : errno:1062 Duplicate entry '14991499' for key 'PRIMARY' con:MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, respHandler=SingleNodeHandler [node=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, packetId=1], host=192.168.31.150, port=3307, statusSync=org.opencloudb.mysql.nio.MySQLConnection$StatusSync@48537945, writeQueue=0, modifiedSQLExecuted=true] frontend host:127.0.0.1/50644/test
再下来开始回滚所有数据结点上的更新操作。
02/03 00:36:10.305 DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=31, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=false, schema=TESTDB]rollback
02/03 00:36:10.305 DEBUG [$_NIOREACTOR-1-RW] (RollbackNodeHandler.java:71) -rollback job run for MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, respHandler=SingleNodeHandler [node=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, packetId=1], host=192.168.31.150, port=3307, statusSync=org.opencloudb.mysql.nio.MySQLConnection$StatusSync@48537945, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.305 DEBUG [$_NIOREACTOR-1-RW] (RollbackNodeHandler.java:71) -rollback job run for MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=dn1{insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1')}, respHandler=SingleNodeHandler [node=dn1{insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1')}, packetId=1], host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.306 DEBUG [$_NIOREACTOR-1-RW] (RollbackNodeHandler.java:71) -rollback job run for MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}, respHandler=SingleNodeHandler [node=dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}, packetId=1], host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:361) -clear session resources org.opencloudb.server.NonBlockingSession@603ec61b
释放数据库链接
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=dn3{insert into t_demo_travel_record (id,context) values (14991499,'这条记录应该进行DN3')}, respHandler=org.opencloudb.mysql.nio.handler.RollbackNodeHandler@394047fa, host=192.168.31.150, port=3307, statusSync=org.opencloudb.mysql.nio.MySQLConnection$StatusSync@48537945, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=dn1{insert into t_demo_travel_record (id,context) values (5001,'这条记录应该进行DN1')}, respHandler=org.opencloudb.mysql.nio.handler.RollbackNodeHandler@394047fa, host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.312 DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=dn2{insert into t_demo_travel_record (id,context) values (5999998,'这条记录应该进行DN2')}, respHandler=org.opencloudb.mysql.nio.handler.RollbackNodeHandler@394047fa, host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (RollbackReleaseHandler.java:58) -autocomit is false,but no commit or rollback ,so mycat rollbacked backend conn MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=org.opencloudb.mysql.nio.handler.RollbackReleaseHandler@2b358b73, host=192.168.31.150, port=3307, statusSync=org.opencloudb.mysql.nio.MySQLConnection$StatusSync@48537945, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=8, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=24, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=null, host=192.168.31.150, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (RollbackReleaseHandler.java:58) -autocomit is false,but no commit or rollback ,so mycat rollbacked backend conn MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=org.opencloudb.mysql.nio.handler.RollbackReleaseHandler@4856412e, host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=16, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=33, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=null, host=192.168.31.187, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (RollbackReleaseHandler.java:58) -autocomit is false,but no commit or rollback ,so mycat rollbacked backend conn MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=org.opencloudb.mysql.nio.handler.RollbackReleaseHandler@16d1581c, host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
02/03 00:36:10.313 DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=28, lastTime=1454430970269, user=root, schema=test, old shema=test, borrowed=true, fromSlaveDB=false, threadId=30, charset=utf8, txIsolation=3, autocommit=false, attachment=null, respHandler=null, host=192.168.31.212, port=3307, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
02
OK，通过2个场景验证了MYCAT的事务一致性。
再补一句，大家可以通过MYSQL权威指南文档了解到其实MYCAT实现一种弱XA的事务控制，怎么理解这个弱XA事务控制。
举例有3个结点，MYCAT根据分片规则依次在不同物理库中预执行，如果当中任何一个发现问题则回滚所有。
若预执行成功，3个结点分别进行commit操作时发生异常，则mycat无法回滚已提交的事务。当然，这种机率很小很小~~~
另外一个问题在于，如果发锁升级导致单结点无法操作时，会对整体应用产生比较大的影响。
本篇完
MyCat 学习笔记 第十二篇.数据分片 之 分片事务处理的更多相关文章
- Django学习笔记第十二篇--关于自定义数据库字段数据类型
一.需求背景: django的models模块提供了很多数据字段的数据类型field,但是总有写奇葩需求不能依靠默认字段满足,所以需要自定义数据数据库数据字段类型.所有的自定义field应该在app路 ...
- MyCat 学习笔记 第十篇.数据分片 之 ER分片
1 应用场景 这篇来说下mycat中自带的er关系分片,所谓er关系分片即可以理解为有关联关系表之间数据分片.类似于订单主表与订单详情表间的分片存储规则. 本文所说的er分片分为两种: a. 依据主键 ...
- 汇编入门学习笔记 （十二）—— int指令、port
疯狂的暑假学习之 汇编入门学习笔记 (十二)-- int指令.port 參考: <汇编语言> 王爽 第13.14章 一.int指令 1. int指令引发的中断 int n指令,相当于引 ...
- VSTO 学习笔记（十二）自定义公式与Ribbon
原文:VSTO 学习笔记(十二)自定义公式与Ribbon 这几天工作中在开发一个Excel插件,包含自定义公式,根据条件从数据库中查询结果.这次我们来做一个简单的测试,达到类似的目的. 即在Excel ...
- Binder学习笔记（十二）—— binder_transaction(...)都干了什么？
binder_open(...)都干了什么? 在回答binder_transaction(...)之前,还有一些基础设施要去探究,比如binder_open(...),binder_mmap(...) ...
- Egret入门学习日记 --- 第十二篇（书中 5.1节 内容）
第十二篇(书中 5.1节 内容) 昨天把 第4章完成了. 今天来看第5章. 接下来是 5.1节 的内容. 总结一下 5.1节 的重点: 1.如何制作一个公用按钮皮肤. 跟着做: 重点1:如何制作一个公 ...
- MySQL数据库学习笔记（十二）----开源工具DbUtils的使用（数据库的增删改查）
[声明] 欢迎转载,但请保留文章原始出处→_→ 生命壹号:http://www.cnblogs.com/smyhvae/ 文章来源:http://www.cnblogs.com/smyhvae/p/4 ...
- java之jvm学习笔记六-十二(实践写自己的安全管理器)(jar包的代码认证和签名) (实践对jar包的代码签名) (策略文件)(策略和保护域) (访问控制器) (访问控制器的栈校验机制) (jvm基本结构)
java之jvm学习笔记六(实践写自己的安全管理器) 安全管理器SecurityManager里设计的内容实在是非常的庞大,它的核心方法就是checkPerssiom这个方法里又调用 AccessCo ...
- Android学习笔记（十二）——实战：制作一个聊天界面
//此系列博文是<第一行Android代码>的学习笔记,如有错漏,欢迎指正! 运用简单的布局知识,我们可以来尝试制作一个聊天界面. 一.制作 Nine-Patch 图片 : Nine-Pa ...
随机推荐
- 如何单独启动wamp 中自带的MySQL
前言:Wamp集成了Apache.MySQL和PHP环境.使用Wamp进行网站开发,是很多网站开发爱好者的选择.同时,其集成的MySQL服务,也常被用于MySQL的开发.这个时候我们只是想启动MySQ ...
- iOS第三方类库汇总【持续更新】
在我们平时开发中会经常使用一些第三方开发的开源类库.这样会有效地提高我们开发项目的效率,在这里我找了好几十个进行一个汇总,供大家参考使用,方便大家在需要的时候能容易找到. UI篇 awesome-io ...
- 常见浏览器兼容问题、盒模型2种模式以及css hack知识讲解
什么是浏览器兼容问题?所谓的浏览器兼容性问题,是指因为不同的浏览器对同一段代码有不同的解析,造成页面显示效果不统一的情况.在大多数情况下,我们的需求是,无论用户用什么浏览器来查看我们的网站或者登陆我们 ...
- HDFS-RAID原理和实现
HDFS-RAID 是Facebook基于hadoop-20-append分支(第一代Hadoop)开发的raid方案,对HDFS的修改极少,主要包括为NameNode增加了根据block信息找到bl ...
- 批处理安装Windows服务，提示"InstallUtil.exe"不是内部命令也不是外部命令解决方式
今天在测试一个C#写的windows服务的时候,在用bat进行调用cmd安装的时候, cd C:\Windows\Microsoft.NET\Framework\v2.0.50727 InstallU ...
- IDEA默认常用快捷键
作为Java的利器,IDEA属实是非常好用,参考网文总结其常用快捷键如下: Ctrl + /(Ctrl + Shift + /):注释或反注释指定的语句.这个是本人最喜欢的,所以排在第一位. Ctrl ...
- Can you explain Lazy Loading?
Introduction Lazy loading is a concept where we delay the loading of the object until the point wher ...
- 重新想象 Windows 8.1 Store Apps (83) - 文件系统的新特性
[源码下载] 重新想象 Windows 8.1 Store Apps (83) - 文件系统的新特性 作者:webabcd 介绍重新想象 Windows 8.1 Store Apps 之文件系统的新特 ...
- fibonacci封闭公式及矩阵连乘
Description The Fibonacci sequence is the sequence of numbers such that every element is equal to th ...
- 那些教程没有的php1-基础知识补漏
php.net 字符串 heredoc结构 类似双引号,其中的变量会被解析.严格遵循下边的格式,结束标识符这行除了可能有一个分号(;)外,绝对不能包含其它字符. <?php $str = < ...