ShardingSphere初探1 --Sharding-JDBC
Sharding-JDBC
引入maven依赖:
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-core</artifactId>
<version>${latest.release.version}</version>
</dependency>
注意将version换为实际的版本号.
简介
Sharding-JDBC是ShardingSphere的第一个产品，也是ShardingSphere的前身。 它定位为轻量级Java框架，在Java的JDBC层提供的额外服务。它使用客户端直连数据库，以jar包形式提供服务，无需额外部署和依赖，可理解为增强版的JDBC驱动，完全兼容JDBC和各种ORM框架。
- 适用于任何基于Java的ORM框架，如：JPA, Hibernate, Mybatis, Spring JDBC Template或直接使用JDBC。
- 基于任何第三方的数据库连接池，如：DBCP, C3P0, BoneCP, Druid, HikariCP等。
- 支持任意实现JDBC规范的数据库。目前支持MySQL，Oracle，SQLServer和PostgreSQL。
对比:
Sharding-JDBC的优势在于对Java应用的友好度.
内部结构
图片解析:
黄色部分
图中黄色部分表示的是Sharding-JDBC的入口API,采用工厂方法的形式提供.
目前有ShardingDataSourceFactory 和 MasterSlaveDataSourceFactory 两个工厂类.
ShardingDataSourceFactory 用于创建分库分表或分库分表+读写分离的JDBC驱动,
MasterSlaveDataSourceFactory用于创建独立使用读写分离的JDBC驱动.
蓝色部分
图中蓝色部分表示的是Sharding-JDBC的配置对象,提供灵活多变的配置方式.
ShardingRuleConfiguration是分库分表配置的核心和入口,它可以包含多个
TableRuleConfiguration 和 MasterSlaveRuleConfiguration .
每一组相同规则分片的表配置一个TableRuleConfiguration. 如果需要分库分表和读写分离共同使用,
每一个读写分离的逻辑库配置一个 MasterSlaveRuleConfiguration .
每个TableRuleConfiguration对应一个ShardingStrategyConfiguration,它有5种实现类可供选择.
* 仅读写分离使用MasterSlaveRuleConfiguration即可.
红色部分
图中红色部分表示的是内部对象,由Sharding-JDBC内部使用,应用开发者无需关注.
Sharding-JDBC通过ShardingRuleConfiguration和MasterSlaveRuleConfiguration生成真正
供ShardingDataSource和MasterSlaveDataSource使用的规则对象.
ShardingDataSource和MasterSlaveDataSource实现了DataSource接口, 是JDBC的完整实现方案.
初始化流程:
1. 配置Configuration对象.
2. 通过Factory对象将Configuration对象转化为Rule对象.
3. 通过Factory对象将Rule对象与DataSource对象装配.
4. Sharding-JDBC使用DataSource对象进行分库.
使用约定
在org.apache.shardingsphere.api和org.apache.shardingsphere.shardingjdbc.api 包中的类是面向用户的API，每次修改都会在release notes中明确声明。 其他包中的类属于内部实现，可能随时进行调整，请勿直接使用。
官方Sharding-JDBC相关使用示例 : https://github.com/apache/incubator-shardingsphere-example
数据分片 什么是数据分片?
首先看不使用Spring的方式:
引入Maven依赖
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-core</artifactId>
<version>${sharding-sphere.version}</version>
</dependency>
基于Java编码的规则配置
Sharding-JDBC的分库分表通过规则配置描述, 以下例子是根据user_id取模分库,且根据order_id取模分表的两库两表的配置.
// 配置真实数据源
Map<String, DataSource> dataSourceMap = new HashMap<>(); // 配置第一个数据源
BasicDataSource dataSource1 = new BasicDataSource();
dataSource1.setDriverClassName("com.mysql.jdbc.Driver");
dataSource1.setUrl("jdbc:mysql://localhost:3306/ds0");
dataSource1.setUsername("root");
dataSource1.setPassword("");
dataSourceMap.put("ds0", dataSource1); // 配置第二个数据源
BasicDataSource dataSource2 = new BasicDataSource();
dataSource2.setDriverClassName("com.mysql.jdbc.Driver");
dataSource2.setUrl("jdbc:mysql://localhost:3306/ds1");
dataSource2.setUsername("root");
dataSource2.setPassword("");
dataSourceMap.put("ds1", dataSource2); // 配置Order表规则
TableRuleConfiguration orderTableRuleConfig = new TableRuleConfiguration();
orderTableRuleConfig.setLogicTable("t_order");
orderTableRuleConfig.setActualDataNodes("ds${0..1}.t_order${0..1}"); // 配置分库 + 分表策略
orderTableRuleConfig.setDatabaseShardingStrategyConfig(new InlineShardingStrategyConfiguration("user_id", "ds${user_id % 2}"));
orderTableRuleConfig.setTableShardingStrategyConfig(new InlineShardingStrategyConfiguration("order_id", "t_order${order_id % 2}")); // 配置分片规则
ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
shardingRuleConfig.getTableRuleConfigs().add(orderTableRuleConfig); // 省略配置order_item表规则...
// ... // 获取数据源对象
DataSource dataSource = ShardingDataSourceFactory.createDataSource(dataSourceMap, shardingRuleConfig, new ConcurrentHashMap(), new Properties());
基于yaml的规则配置:
或通过Yaml方式配置,与以上配置等价:
dataSources:
ds0: !!org.apache.commons.dbcp.BasicDataSource
driverClassName: com.mysql.jdbc.Driver
url: jdbc:mysql://localhost:3306/ds0
username: root
password:
ds1: !!org.apache.commons.dbcp.BasicDataSource
driverClassName: com.mysql.jdbc.Driver
url: jdbc:mysql://localhost:3306/ds1
username: root
password: shardingRule:
tables:
t_order:
actualDataNodes: ds${0..1}.t_order${0..1}
databaseStrategy:
inline:
shardingColumn: user_id
algorithmInlineExpression: ds${user_id % 2}
tableStrategy:
inline:
shardingColumn: order_id
algorithmInlineExpression: t_order${order_id % 2}
t_order_item:
actualDataNodes: ds${0..1}.t_order_item${0..1}
databaseStrategy:
inline:
shardingColumn: user_id
algorithmInlineExpression: ds${user_id % 2}
tableStrategy:
inline:
shardingColumn: order_id
algorithmInlineExpression: t_order_item${order_id % 2}
DataSource dataSource = YamlShardingDataSourceFactory.createDataSource(yamlFile);
使用原生JDBC:
通过ShardingDataSourceFactory或者YamlShardingDataSourceFactory工厂和规则配置对象获取ShardingDataSource, ShardingDataSource实现自JDBC的
标准接口DataSource. 然后可通过DataSource选择使用原生JDBC开发, 或者使用JPA, MyBatis等ORM工具. 以JDBC原生实现为例:
DataSource dataSource = YamlShardingDataSourceFactory.createDataSource(yamlFile);
String sql = "SELECT i.* FROM t_order o JOIN t_order_item i ON o.order_id=i.order_id WHERE o.user_id=? AND o.order_id=?";
try (
Connection conn = dataSource.getConnection();
PreparedStatement preparedStatement = conn.prepareStatement(sql)) {
preparedStatement.setInt(1, 10);
preparedStatement.setInt(2, 1001);
try (ResultSet rs = preparedStatement.executeQuery()) {
while(rs.next()) {
System.out.println(rs.getInt(1));
System.out.println(rs.getInt(2));
}
}
}
使用Spring
引入Maven依赖
<!-- for spring boot -->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>${sharding-sphere.version}</version>
</dependency> <!-- for spring namespace -->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-namespace</artifactId>
<version>${sharding-sphere.version}</version>
</dependency>
基于Spring boot的规则配置
spring.shardingsphere.datasource.names=ds0,ds1 spring.shardingsphere.datasource.ds0.type=org.apache.commons.dbcp2.BasicDataSource
spring.shardingsphere.datasource.ds0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds0.url=jdbc:mysql://localhost:3306/ds0
spring.shardingsphere.datasource.ds0.username=root
spring.shardingsphere.datasource.ds0.password= spring.shardingsphere.datasource.ds1.type=org.apache.commons.dbcp2.BasicDataSource
spring.shardingsphere.datasource.ds1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds1.url=jdbc:mysql://localhost:3306/ds1
spring.shardingsphere.datasource.ds1.username=root
spring.shardingsphere.datasource.ds1.password= spring.shardingsphere.sharding.default-database-strategy.inline.sharding-column=user_id
spring.shardingsphere.sharding.default-database-strategy.inline.algorithm-expression=ds$->{user_id % 2} spring.shardingsphere.sharding.tables.t_order.actual-data-nodes=ds$->{0..1}.t_order$->{0..1}
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.algorithm-expression=t_order$->{order_id % 2} spring.shardingsphere.sharding.tables.t_order_item.actual-data-nodes=ds$->{0..1}.t_order_item$->{0..1}
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.algorithm-expression=t_order_item$->{order_id % 2}
基于Spring boot + JNDI的规则配置
如果您计划使用Spring boot + JNDI的方式, 在应用容器(如Tomcat)中使用Sharding-JDBC时,可使用
spring.shardingsphere.datasource.${datasourceName}.jndiName来替代数据源的一系列配置.如:
spring.shardingsphere.datasource.names=ds0,ds1 spring.shardingsphere.datasource.ds0.jndi-name=java:comp/env/jdbc/ds0
spring.shardingsphere.datasource.ds1.jndi-name=jdbc/ds1 spring.shardingsphere.sharding.default-database-strategy.inline.sharding-column=user_id
spring.shardingsphere.sharding.default-database-strategy.inline.algorithm-expression=ds$->{user_id % 2} spring.shardingsphere.sharding.tables.t_order.actual-data-nodes=ds$->{0..1}.t_order$->{0..1}
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.algorithm-expression=t_order$->{order_id % 2} spring.shardingsphere.sharding.tables.t_order_item.actual-data-nodes=ds$->{0..1}.t_order_item$->{0..1}
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.algorithm-expression=t_order_item$->{order_id % 2}
基于Spring命名空间的规则配置:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:sharding="http://shardingsphere.apache.org/schema/shardingsphere/sharding"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://shardingsphere.apache.org/schema/shardingsphere/sharding
http://shardingsphere.apache.org/schema/shardingsphere/sharding/sharding.xsd
">
<bean id="ds0" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/ds0" />
<property name="username" value="root" />
<property name="password" value="" />
</bean>
<bean id="ds1" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/ds1" />
<property name="username" value="root" />
<property name="password" value="" />
</bean> <sharding:inline-strategy id="databaseStrategy" sharding-column="user_id" algorithm-expression="ds$->{user_id % 2}" />
<sharding:inline-strategy id="orderTableStrategy" sharding-column="order_id" algorithm-expression="t_order$->{order_id % 2}" />
<sharding:inline-strategy id="orderItemTableStrategy" sharding-column="order_id" algorithm-expression="t_order_item$->{order_id % 2}" /> <sharding:data-source id="shardingDataSource">
<sharding:sharding-rule data-source-names="ds0,ds1">
<sharding:table-rules>
<sharding:table-rule logic-table="t_order" actual-data-nodes="ds$->{0..1}.t_order$->{0..1}" database-strategy-ref="databaseStrategy" table-strategy-ref="orderTableStrategy" />
<sharding:table-rule logic-table="t_order_item" actual-data-nodes="ds$->{0..1}.t_order_item$->{0..1}" database-strategy-ref="databaseStrategy" table-strategy-ref="orderItemTableStrategy" />
</sharding:table-rules>
</sharding:sharding-rule>
</sharding:data-source>
</beans>
在Spring中使用DataSource
直接通过注入的方式即可使用DataSource, 或者将DataSource配置在JPA,Hibernate或MyBatis中使用.
@Resource
private DataSource dataSource;
规则配置包括数据源配置,表规则配置,分库策略和分表策略组成.这只是最简单的配置方式,实际使用可更加灵活,
如:多分片键, 分片策略直接和表规则配置绑定等.
读写分离
同样读写分离的也是单独的依赖
不使用Spring
引入Maven依赖:
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-core</artifactId>
<version>${sharding-sphere.version}</version>
</dependency>
基于Java编码的规则配置
// 配置真实数据源
Map<String, DataSource> dataSourceMap = new HashMap<>(); // 配置主库
BasicDataSource masterDataSource = new BasicDataSource();
masterDataSource.setDriverClassName("com.mysql.jdbc.Driver");
masterDataSource.setUrl("jdbc:mysql://localhost:3306/ds_master");
masterDataSource.setUsername("root");
masterDataSource.setPassword("");
dataSourceMap.put("ds_master", masterDataSource); // 配置第一个从库
BasicDataSource slaveDataSource1 = new BasicDataSource();
slaveDataSource1.setDriverClassName("com.mysql.jdbc.Driver");
slaveDataSource1.setUrl("jdbc:mysql://localhost:3306/ds_slave0");
slaveDataSource1.setUsername("root");
slaveDataSource1.setPassword("");
dataSourceMap.put("ds_slave0", slaveDataSource1); // 配置第二个从库
BasicDataSource slaveDataSource2 = new BasicDataSource();
slaveDataSource2.setDriverClassName("com.mysql.jdbc.Driver");
slaveDataSource2.setUrl("jdbc:mysql://localhost:3306/ds_slave1");
slaveDataSource2.setUsername("root");
slaveDataSource2.setPassword("");
dataSourceMap.put("ds_slave1", slaveDataSource2); // 配置读写分离规则
MasterSlaveRuleConfiguration masterSlaveRuleConfig = new MasterSlaveRuleConfiguration("ds_master_slave", "ds_master", Arrays.asList("ds_slave0", "ds_slave1")); // 获取数据源对象
DataSource dataSource = MasterSlaveDataSourceFactory.createDataSource(createDataSourceMap(), masterSlaveRuleConfig, new HashMap<String, Object>(), new Properties());
基于Yaml的规则配置:
或通过Yaml方式配置,与以上配置等价:
dataSources:
ds_master: !!org.apache.commons.dbcp.BasicDataSource
driverClassName: com.mysql.jdbc.Driver
url: jdbc:mysql://localhost:3306/ds_master
username: root
password:
ds_slave0: !!org.apache.commons.dbcp.BasicDataSource
driverClassName: com.mysql.jdbc.Driver
url: jdbc:mysql://localhost:3306/ds_slave0
username: root
password:
ds_slave1: !!org.apache.commons.dbcp.BasicDataSource
driverClassName: com.mysql.jdbc.Driver
url: jdbc:mysql://localhost:3306/ds_slave1
username: root
password: masterSlaveRule:
name: ds_ms
masterDataSourceName: ds_master
slaveDataSourceNames: [ds_slave0, ds_slave1] props:
sql.show: true
DataSource dataSource = MasterSlaveDataSourceFactory.createDataSource(yamlFile);
使用原生JDBC
通过MasterSlaveDataSourceFactory工厂和规则配置对象获取MasterSlaveDataSource, MasterSlaveDataSource实现自JDBC的标准接口DataSource.
然后可通过DataSource选择使用原生JDBC开发,或者使用JPA, MyBatis等ORM工具. 以JDBC原生实现为例:
DataSource dataSource = MasterSlaveDataSourceFactory.createDataSource(yamlFile);
String sql = "SELECT i.* FROM t_order o JOIN t_order_item i ON o.order_id=i.order_id WHERE o.user_id=? AND o.order_id=?";
try (
Connection conn = dataSource.getConnection();
PreparedStatement preparedStatement = conn.prepareStatement(sql)) {
preparedStatement.setInt(1, 10);
preparedStatement.setInt(2, 1001);
try (ResultSet rs = preparedStatement.executeQuery()) {
while(rs.next()) {
System.out.println(rs.getInt(1));
System.out.println(rs.getInt(2));
}
}
}
使用Spring
引入Maven依赖:
<!-- for spring boot -->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>${sharding-sphere.version}</version>
</dependency> <!-- for spring namespace -->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-namespace</artifactId>
<version>${sharding-sphere.version}</version>
</dependency>
基于Spring boot的规则配置:
spring.shardingsphere.datasource.names=master,slave0,slave1 spring.shardingsphere.datasource.master.type=org.apache.commons.dbcp.BasicDataSource
spring.shardingsphere.datasource.master.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.master.url=jdbc:mysql://localhost:3306/master
spring.shardingsphere.datasource.master.username=root
spring.shardingsphere.datasource.master.password= spring.shardingsphere.datasource.slave0.type=org.apache.commons.dbcp.BasicDataSource
spring.shardingsphere.datasource.slave0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.slave0.url=jdbc:mysql://localhost:3306/slave0
spring.shardingsphere.datasource.slave0.username=root
spring.shardingsphere.datasource.slave0.password= spring.shardingsphere.datasource.slave1.type=org.apache.commons.dbcp.BasicDataSource
spring.shardingsphere.datasource.slave1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.slave1.url=jdbc:mysql://localhost:3306/slave1
spring.shardingsphere.datasource.slave1.username=root
spring.shardingsphere.datasource.slave1.password= spring.shardingsphere.masterslave.name=ms
spring.shardingsphere.masterslave.master-data-source-name=master
spring.shardingsphere.masterslave.slave-data-source-names=slave0,slave1 spring.shardingsphere.props.sql.show=true
基于Spring boot + JNDI的规则配置:
如果您计划使用Spring boot + JNDI的方式, 在应用容器(如Tomcat)中使用Sharding-JDBC时,可使用spring.shardingsphere.datasource.${datasourceName}.jndiName来代替数据源的一系列配置.如:
spring.shardingsphere.datasource.names=master,slave0,slave1 spring.shardingsphere.datasource.master.jndi-name=java:comp/env/jdbc/master
spring.shardingsphere.datasource.slave0.jndi-name=jdbc/slave0
spring.shardingsphere.datasource.slave1.jndi-name=jdbc/slave1 spring.shardingsphere.masterslave.name=ms
spring.shardingsphere.masterslave.master-data-source-name=master
spring.shardingsphere.masterslave.slave-data-source-names=slave0,slave1 spring.shardingsphere.props.sql.show=true
基于Spring命名空间的规则配置:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:master-slave="http://shardingsphere.apache.org/schema/shardingsphere/masterslave"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://shardingsphere.apache.org/schema/shardingsphere/masterslave
http://shardingsphere.apache.org/schema/shardingsphere/masterslave/master-slave.xsd
">
<bean id="ds_master" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/ds_master" />
<property name="username" value="root" />
<property name="password" value="" />
</bean>
<bean id="ds_slave0" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/ds_slave0" />
<property name="username" value="root" />
<property name="password" value="" />
</bean>
<bean id="ds_slave1" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/ds_slave1" />
<property name="username" value="root" />
<property name="password" value="" />
</bean> <master-slave:data-source id="masterSlaveDataSource" master-data-source-name="ds_master" slave-data-source-names="ds_slave0, ds_slave1" >
<master-slave:props>
<prop key="sql.show">${sql_show}</prop>
<prop key="executor.size">10</prop>
<prop key="foo">bar</prop>
</master-slave:props>
</master-slave:data-source>
</beans>
在Spring中使用DataSource
直接通过注入的方式即可使用DataSource, 或者将DataSource配置在JPA, Hibernate或MyBatis中使用.
@Resource
private DataSource dataSource;
配置:
配置是整个Sharding-JDBC的核心,是Sharding-JDBC中唯一与应用开发者打交道的模块. 配置模块也是Sharding-JDBC的门户,
通过它可以快速清晰的理解Sharding-JDBC所提供的功能.
本部分是Sharding-JDBC的配置参考手册,需要时可当做字典查阅.
Sharding-JDBC提供了4种配置方式,用于不同的使用场景.通过配置,应用开发者可以灵活的使用分库分表,读写分离以及分库分表+读写分离共用.
强制路由
ShardingSphere使用ThreadLocal管理分片键值进行Hint强制路由. 可以通过编程的方式向HintManager中添加分片值,该分片值仅在当前线程内生效.
Hint方式主要使用场景:
1. 分片字段不存在SQL中, 数据库表结构中, 而存在于外部业务逻辑.
2. 强制在主库进行某些数据操作.
基于暗示(Hint)的数据分片
配置Hint分片算法
Hint分片算法需要用户实现org.apache.shardingsphere.api.sharding.hint.HintShardingAlgorithm接口.
ShardingSphere在进行Routing时,如果发现LogicTable的TableRule采用了Hint的分片算法,将会从HintManager中获取分片值进行路由操作.
参考配置如下:
shardingRule:
tables:
t_order:
actualDataNodes: demo_ds_${0..1}.t_order_${0..1}
databaseStrategy:
hint:
algorithmClassName: org.apache.shardingsphere.userAlgo.HintAlgorithm
tableStrategy:
hint:
algorithmClassName: org.apache.shardingsphere.userAlgo.HintAlgorithm
defaultTableStrategy:
none:
defaultKeyGenerator:
type: SNOWFLAKE
column: order_id
props:
sql.show: true
获取HintManager
HintManager hintManager = HintManager.getInstance();
添加分片键值
* 使用hintManager.addDatabaseShardingValue来添加数据源分片键值.
* 使用hintManager.addTableShardingValue来添加表分片键值.
分库不分表情况下, 强制路由至某一个分库时, 可使用hintManager.setDatabaseShardingValue方式添加分片.
通过此方式添加分片键值后,将跳过SQL解析和改写阶段, 从而提高整体执行效率.
清除分片键值
分片键值保存存在ThreadLocal中,所以需要在操作结束时调用hintManager.close()来清除ThreadLocal中的内容.
hintManager实现了AutoCloseable接口, 可推荐使用 try with resource 自动关闭.
完整代码示例:
// Sharding database and table with using hintManager.
String sql = "SELECT * FROM t_order";
try (HintManager hintManager = HintManager.getInstance();
Connection conn = dataSource.getConnection();
PreparedStatement preparedStatement = conn.prepareStatement(sql)) {
hintManager.addDatabaseShardingValue("t_order", 1);
hintManager.addTableShardingValue("t_order", 2);
try (ResultSet rs = preparedStatement.executeQuery()) {
while (rs.next()) {
// ...
}
}
} // Sharding database without sharding table and routing to only one database with using hintManger.
String sql = "SELECT * FROM t_order";
try (HintManager hintManager = HintManager.getInstance();
Connection conn = dataSource.getConnection();
PreparedStatement preparedStatement = conn.prepareStatement(sql)) {
hintManager.setDatabaseShardingValue(3);
try (ResultSet rs = preparedStatement.executeQuery()) {
while (rs.next()) {
// ...
}
}
}
基于暗示(Hint)的强制主库路由
获取HintManager
与基于暗示(Hint)的数据分片相同.
设置主库路由
* 使用hintManager.setMasterRouteOnly设置主库路由.
清除分片键值
与基于暗示(Hint)的数据分片相同.
完整代码示例:
String sql = "SELECT * FROM t_order";
try (
HintManager hintManager = HintManager.getInstance();
Connection conn = dataSource.getConnection();
PreparedStatement preparedStatement = conn.prepareStatement(sql)) {
hintManager.setMasterRouteOnly();
try (ResultSet rs = preparedStatement.executeQuery()) {
while (rs.next()) {
// ...
}
}
}
编排治理(想起了mooc服务计算与服务生态系统)
使用治理功能需要指定一个注册中心. 配置将全部存入注册中心, 可以在每次启动时使用本地配置覆盖注册中心配置,
也可以只通过注册中心读取配置.
不使用Spring
引入Maven依赖:
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-orchestration</artifactId>
<version>${sharding-sphere.version}</version>
</dependency> <!--若使用zookeeper, 请加入下面Maven坐标-->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-orchestration-reg-zookeeper-curator</artifactId>
</dependency> <!--若使用etcd, 请下面Maven坐标-->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-orchestration-reg-etcd</artifactId>
</dependency>
基于Java编码的规则配置:
// 省略配置dataSourceMap以及shardingRuleConfig
// ... // 配置注册中心
RegistryCenterConfiguration regConfig = new RegistryCenterConfiguration();
regConfig.setServerLists("localhost:2181");
regConfig.setNamespace("sharding-sphere-orchestration"); // 配置治理
OrchestrationConfiguration orchConfig = new OrchestrationConfiguration("orchestration-sharding-data-source", regConfig, false); // 获取数据源对象
DataSource dataSource = OrchestrationShardingDataSourceFactory.createDataSource(dataSourceMap, shardingRuleConfig, new ConcurrentHashMap(), new Properties(), orchConfig);
基于Yaml的规则配置
或通过Yaml方式配置,与以上配置等价:
orchestration:
name: orchestration-sharding-data-source
overwrite: false
registry:
serverLists: localhost:2181
namespace: sharding-sphere-orchestration
DataSource dataSource = YamlOrchestrationShardingDataSourceFactory.createDataSource(yamlFile);
使用Spring
引入Maven依赖:
<!-- for spring boot -->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-orchestration-spring-boot-starter</artifactId>
<version>${sharding-sphere.version}</version>
</dependency> <!--若使用zookeeper, 请加入下面Maven坐标-->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-orchestration-reg-zookeeper-curator</artifactId>
<version>${sharding-sphere.version}</version>
</dependency> <!--若使用etcd, 请加入下面Maven坐标-->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-orchestration-reg-etcd</artifactId>
<version>${sharding-sphere.version}</version>
</dependency>
<!-- for spring namespace -->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-orchestration-spring-namespace</artifactId>
<version>${sharding-sphere.version}</version>
</dependency> <!--若使用zookeeper, 请加入下面Maven坐标-->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-orchestration-reg-zookeeper-curator</artifactId>
<version>${sharding-sphere.version}</version>
</dependency> <!--若使用etcd, 请加入下面Maven坐标-->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-orchestration-reg-etcd</artifactId>
<version>${sharding-sphere.version}</version>
</dependency>
基于Spring boot的规则配置
spring.shardingsphere.orchestration.name=orchestration-sharding-data-source
spring.shardingsphere.orchestration.overwrite=false
spring.shardingsphere.orchestration.registry.server-lists=localhost:2181
spring.shardingsphere.orchestration.registry.namespace=sharding-jdbc-orchestration
基于Spring命名空间的规则配置:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:orchestraion="http://shardingsphere.apache.org/schema/shardingsphere/orchestration"
xmlns="http://www.springframework.org/schema/beans"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://shardingsphere.apache.org/schema/shardingsphere/orchestration
http://shardingsphere.apache.org/schema/shardingsphere/orchestration/orchestration.xsd">
<import resource="namespace/shardingDataSourceNamespace.xml" />
<orchestraion:registry-center id="regCenter" server-lists="localhost:3181" namespace="orchestration-spring-namespace-test" operation-timeout-milliseconds="1000" max-retries="3" />
<orchestraion:sharding-data-source id="simpleShardingOrchestration" data-source-ref="simpleShardingDataSource" registry-center-ref="regCenter" />
</beans>
分布式事务
两阶段提交-XA
引入Maven依赖
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-transaction-xa-core</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
XA事务管理器将以SPI的方式被Sharding-JDBC所加载.
Atomikos参数配置(可选)
ShardingSphere默认的XA事务管理器为Atomikos, 在项目的logs目录中会生成 xa_tx.log,
这是XA崩溃恢复时所需的日志,请勿删除.
也可以通过在项目的classpath中添加 jta.properties 来订制化Atomikos配置项.
第三方BASE实现-Saga
目前Apache/incubator-shardingsphere暂无BASE事务的实现,但是仍然可以使用第三方实现的Saga事务.
https://shardingsphere.apache.org/document/current/cn/manual/sharding-jdbc/usage/transaction/
ShardingSphere初探1 --Sharding-JDBC的更多相关文章
- spring boot:配置shardingsphere(sharding jdbc)使用druid数据源(druid 1.1.23 / sharding-jdbc 4.1.1 / mybatis / spring boot 2.3.3)
一,为什么要使用druid数据源? 1,druid的优点 Druid是阿里巴巴开发的号称为监控而生的数据库连接池 它的优点包括: 可以监控数据库访问性能 SQL执行日志 SQL防火墙 但spring ...
- sharding jdbc(sphere) 3.1.0 spring boot配置
sharding jdbc 2.x系列详解参见https://www.cnblogs.com/zhjh256/p/9221634.html. 最近将sharding jdbc的配置从xml切换到了sp ...
- Sharding JDBC整合SpringBoot 2.x 和 MyBatis Plus 进行分库分表
Sharding JDBC整合SpringBoot 2.x 和 MyBatis Plus 进行分库分表 交易所流水表的单表数据量已经过亿,选用Sharding-JDBC进行分库分表.MyBatis-P ...
- Sharding jdbc 强制路由策略（HintShardingStrategy）使用记录
背景 随着项目运行时间逐渐增加,数据库中的数据也越来越多,虽然加索引,优化查询,但是数据量太大,还是会影响查询效率,也给数据库增加了负载. 再加上冷数据基本不使用的场景,决定采用分表来处理数据,从而来 ...
- Sharding JDBC案例实战
基础分库 以下实例基于shardingsphere 4.1.0 + SpringBoot 2.2.5.RELEASE版本 依赖导入: <properties> <project.bu ...
- Spring boot项目集成Sharding Jdbc
环境 jdk:1.8 framework: spring boot, sharding jdbc database: MySQL 搭建步骤 在pom 中加入sharding 依赖 <depend ...
- 记录因Sharding Jdbc批量操作引发的一次fullGC
周五晚上告警群突然收到了一条告警消息,点开一看,应用 fullGC 了. 于是赶紧联系运维下载堆内存快照,进行分析. 内存分析 使用 MemoryAnalyzer 打开堆文件 mat 下载地址:htt ...
- 浅谈sharding jdbc
定位为轻量级Java框架,在Java的JDBC层提供的额外服务. 它使用客户端直连数据库,以jar包形式提供服务,无需额外部署和依赖,可理解为增强版的JDBC驱动,完全兼容JDBC和各种ORM框架. ...
- ShardingSphere初探1 -- 概览
知道这个框架是通过一期QQ课堂 https://shardingsphere.apache.org 官网 https://github.com/apache/incubator-shardingsph ...
随机推荐
- mybatis逆向生成dao mapper和example.java文件
mabatis插件 <plugin> <groupId>org.mybatis.generator</groupId> <artifactId>myba ...
- React+SpringBoot项目部署
静态资源访问配置 https://www.jianshu.com/p/b6e0a0df32ec https://segmentfault.com/q/1010000012240531/a-102000 ...
- java之基本技术点总结博客
泛型的理解 聊一聊-JAVA 泛型中的通配符 T,E,K,V,? 类,接口的继承和实现的规则 类与类之间只能继承,并且是单继承,可以多级继承 类与接口之间可以实现,一个类可以实现多个接口 接口和接口之 ...
- 中控考勤机使用 zkemkeeper SDK订阅考勤数据事件失效解决方式
问题 前同事编写的对中控考勤机数据集成项目当中,打卡数据不能实时进行上传到平台当中,一直靠定时全量上传来同步数据. 阅读代码后,发现代码中有实时上传数据的逻辑,但是运行一段时间后,中控zkemkeep ...
- C# vb .net实现博物馆哑色框架特效滤镜
在.net中,如何简单快捷地实现Photoshop滤镜组中的博物馆哑色框架效果呢?答案是调用SharpImage!专业图像特效滤镜和合成类库.下面开始演示关键代码,您也可以在文末下载全部源码: 设置授 ...
- 4.matplotlib绘制直方图
  # coding=utf-8 from matplotlib import pyplot as plt from matplotlib import font_manager a=[131, ...
- Java中Date、String、Calendar类型之间的转化
1.Calendar 转化 String //获取当前时间的具体情况,如年,月,日,week,date,分,秒等 Calendar calendat = Calendar.getInstanc ...
- InputStream和OutputStream及相关知识汇总
https://www.jianshu.com/p/e5bc7ea5f948 最近帮学姐写爬虫的时候遇到奇怪的问题,同样的程序在Mac上可以正常运行而在Windows上返回结果错误,最后经排查发现是L ...
- IDEA中Tomcat热部署不生效问题解决办法
IDEA中Tomcat热部署不生效问题解决办法 1.设置完热部署后 2.一定要在Debug模式下运行不要点RUN!!!!!!!!!!!!!!!!!
- JavaScript_proto_和prototype到底是什么玩意
_proto_和prototype到底有什么区别啊?是个什么东西啊? 在这里我头也比较大啊,小学语文没学好,所以组织能力比较差劲,所以尽量的咱用代码来解释吧. function too() { thi ...