目录

  一、Sharding-JDBC依赖

  二、代码实践

  三、源码分析

在上一篇博文中,介绍了Sharding-JDBC的分片策略、分片键和分片算法的基本概念,以及2.0.3版本可以支持和无法支持的使用场景。

可以支持的场景:支持对SQL语句中的=、IN和BETWEEN AND的分片操作,但前提是分片键必须存在于SQL和数据表结构中。

无法支持的场景:分片键不存在于SQL和数据表结构中,即基于暗示(Hint)的数据分片操作(2.0.3版本的问题)。

无可厚非,缺少了Hint分片策略的支持,Sharding-JDBC 2.0.3版本的使用场景就非常受限了,但值得庆幸的是,此问题在3.x版本进行了修复(这里可以有掌声!),接下来的代码皆基于3.1.0版本。

一、Sharding-JDBC依赖

<!-- sharding-jdbc-core -->
<dependency>
<groupId>io.shardingsphere</groupId>
<artifactId>sharding-jdbc-core</artifactId>
<version>3.1.0</version>
</dependency>
<!-- sharding-jdbc-spring-namespace -->
<dependency>
<groupId>io.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-namespace</artifactId>
<version>3.1.0</version>
</dependency>

和2.0.3版本相比,依赖的名称有所改变,不要搞错了哦。

二、代码实践

业务背景就不再介绍了,不了解可移步至Sharding-JDBC(二)2.0.3版本实践

如下代码配置了标准分片策略中的精确分片算法PreciseShardingAlgorithm和Hint分片算法HintShardingAlgorithm。

XML配置:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:sharding="http://shardingsphere.io/schema/shardingsphere/sharding"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx.xsd
http://shardingsphere.io/schema/shardingsphere/sharding
http://shardingsphere.io/schema/shardingsphere/sharding/sharding.xsd"> <!-- 标准分片策略 -->
<sharding:standard-strategy id="settlementTableShardingStandardStrategy" sharding-column="pay_serial_number"
precise-algorithm-ref="preciseTableShardingAlgorithm"/> <!-- 基于暗示(Hint)的分片策略 -->
<sharding:hint-strategy id="settlementHintTableShardingStrategy" algorithm-ref="hintTableShardingAlgorithm"/>
<sharding:hint-strategy id="settlementHintDatabaseShardingStrategy" algorithm-ref="hintDatabaseShardingAlgorithm"/> <sharding:data-source id="shardingDataSource">
<sharding:sharding-rule data-source-names="dataSource">
<sharding:table-rules>
<sharding:table-rule logic-table="settlement"
table-strategy-ref="settlementTableShardingStandardStrategy"/>
<!-- logic-table参数的大小写必须和SettlementMapper.xml中selectByExample方法的表名大小一致!!! -->
<!-- logic-table必须和org.cellphone.finance.repo.SettlementRepository.querySettlements中的logicTable及SQL中的表名一致,否则无法找到分片策略 -->
<!-- 逻辑表名,不需要和真实表名一致 -->
<sharding:table-rule logic-table="settlement_hint"
database-strategy-ref="settlementHintDatabaseShardingStrategy"
table-strategy-ref="settlementHintTableShardingStrategy"/>
</sharding:table-rules>
</sharding:sharding-rule>
<sharding:props>
<prop key="sql.show">true</prop>
</sharding:props>
</sharding:data-source> <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="shardingDataSource"/>
</bean>
<tx:annotation-driven/> </beans>

精确分片算法:

package org.cellphone.finance.repo.sharding;

import com.google.common.collect.Lists;
import io.shardingsphere.api.algorithm.sharding.PreciseShardingValue;
import io.shardingsphere.api.algorithm.sharding.standard.PreciseShardingAlgorithm;
import org.apache.commons.collections.CollectionUtils;
import org.cellphone.common.constant.CommonConst;
import org.springframework.stereotype.Component; import java.util.Collection; /**
* 精确分片算法,属于标准分片算法,用于处理=和IN的分片
* <p>
* 使用精确分片算法的前提:分片字段必须存在与SQL中、数据库表结构中,否则无法使用精确分片算法
* <p>
* 此分片算法应用于SETTLEMENT数据表,这里是按天分表
* <p>
* 特别说明:Sharding Jdbc版本:3.1.0
* <p>
* Created by on 2018/4/9.
*/
@Component("preciseTableShardingAlgorithm")
public class PreciseTableShardingAlgorithm implements PreciseShardingAlgorithm<String> { /**
* 精确分片算法
*
* @param availableTargetNames 目标数据源名称或数据表名称,注意:是逻辑数据源名或逻辑数据表名,来自SQL
* @param shardingValue 分片值,来自SQL中分片字段对应的值
* @return 真实数据源名称或数据表名称
*/
@Override
public String doSharding(final Collection<String> availableTargetNames, final PreciseShardingValue<String> shardingValue) {
// 默认数据表名称,有可能数据库中不存在这张表
String tableName = "settlement"; // 逻辑表名为空,返回默认表名
if (CollectionUtils.isEmpty(availableTargetNames))
return tableName; // availableTargetNames来自SQL,只有一个元素
tableName = Lists.newArrayList(availableTargetNames).get(0); String paySerialNumber = shardingValue.getValue();
String suffix = paySerialNumber.substring(5, 13);
return tableName + CommonConst.UNDERLINE + suffix;
}
}

Hint数据源分片算法:

package org.cellphone.finance.repo.sharding;

import io.shardingsphere.api.algorithm.sharding.ShardingValue;
import io.shardingsphere.api.algorithm.sharding.hint.HintShardingAlgorithm;
import org.springframework.stereotype.Component; import java.util.Collection; /**
* Sharding Jdbc基于暗示(Hint)的数据分片算法
*
* 使用Sharding Jdbc 3.x版本时,此数据源分片算法这个一定要有!!!
* 否则无法正常使用org.cellphone.finance.repo.sharding.HintTableShardingAlgorithm算法
* <p>
* Created by on 2019/4/25.
*/
@Component("hintDatabaseShardingAlgorithm")
public class HintDatabaseShardingAlgorithm implements HintShardingAlgorithm { @Override
public Collection<String> doSharding(Collection<String> availableTargetNames, ShardingValue shardingValue) {
return availableTargetNames;
}
}

Hint数据表分片算法:

package org.cellphone.finance.repo.sharding;

import com.google.common.collect.Lists;
import io.shardingsphere.api.algorithm.sharding.ListShardingValue;
import io.shardingsphere.api.algorithm.sharding.ShardingValue;
import io.shardingsphere.api.algorithm.sharding.hint.HintShardingAlgorithm;
import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.time.DateFormatUtils;
import org.apache.commons.lang3.time.DateUtils;
import org.cellphone.common.constant.CommonConst;
import org.cellphone.common.constant.DateConst;
import org.springframework.stereotype.Component; import java.text.ParseException;
import java.util.*; /**
* Sharding Jdbc基于暗示(Hint)的数据分片算法
* 版本:Sharding Jdbc 3.1.0
*
* 官方介绍(2.x版本):http://shardingsphere.apache.org/document/legacy/2.x/cn/02-guide/hint-sharding-value/
* 官方介绍(4.x版本):https://shardingsphere.apache.org/document/current/cn/manual/sharding-jdbc/usage/hint/
* <p>
* <p>
* <p>
* <p>
* 使用此算法的背景如下:
* 1. SETTLEMENT(支付表)是按时间维度进行分表,该时间取自PAY_SERIAL_NUMBER中的时间数据,非表中的时间字段;
* <p>
* 2. 使用用户手机号此类条件查询数据时,请求参数除传入业务参数外,
* 还需传入时间段(即分片字段,例:startTime和endTime)以确定分表范围;
* 但是!!!startTime和endTime(即分片字段)不存在SQL中、数据库表结构中,而存在于外部业务逻辑
* <p>
* <p>
* <p>
* 因此,第2点导致无法直接使用Sharding Jdbc的PreciseShardingAlgorithm(精确分片算法)或RangeShardingAlgorithm(范围分片算法),
* 只能使用HintShardingAlgorithm(基于暗示的数据分片算法),该算法的使用场景如下:
* 1. 分片字段不存在SQL中、数据库表结构中,而存在于外部业务逻辑;
* <p>
* 2. 强制在主库进行某些数据操作。
* <p>
* <p>
* Created by on 2019/4/25.
*/
@Component("hintTableShardingAlgorithm")
public class HintTableShardingAlgorithm implements HintShardingAlgorithm { /**
* 分片算法
*
* @param availableTargetNames 逻辑数据库名称或逻辑数据表名称
* @param shardingValue 用来确定分表的参数
* @return 实际数据表名称列表,SQL实际操作的数据表
*/
@Override
public Collection<String> doSharding(Collection<String> availableTargetNames, ShardingValue shardingValue) { String realTableName = StringUtils.EMPTY;
for (String each : availableTargetNames) {
if (StringUtils.isNotBlank(each)) {
// 基于hint的逻辑表名:settlement_hint
realTableName = each.replace("_hint", StringUtils.EMPTY);
break;
}
} List<String> tables = new ArrayList<>(); ListShardingValue<String> listShardingValue = (ListShardingValue<String>) shardingValue;
List<String> list = Lists.newArrayList(listShardingValue.getValues()); // 缺少确定分表的参数,无法确定具体分表,直接返回真实表名称
if (CollectionUtils.isEmpty(list)) {
tables.add(realTableName);
return tables;
} // 拆分分表参数,此参数值来自:com.fcbox.manage.core.repo.FcBoxPostRepository.queryFcBoxPosts()
String[] queryTime = list.get(0).split(CommonConst.UNDERLINE);
Date startTime, endTime;
try {
startTime = DateUtils.parseDate(queryTime[0], DateConst.DATE_FORMAT_NORMAL);
endTime = DateUtils.parseDate(queryTime[1], DateConst.DATE_FORMAT_NORMAL);
} catch (ParseException e) {
// 分表参数解析错误,无法确定具体分表,直接返回真实表名称
tables.add(realTableName);
return tables;
} Calendar calendar = Calendar.getInstance();
// 组织startTime和endTime时段范围内的分表
while (startTime.getTime() <= endTime.getTime()) {
tables.add(realTableName + CommonConst.UNDERLINE + DateFormatUtils.format(startTime, DateConst.DATE_FORMAT_YYYY_MM_DD));
calendar.setTime(startTime);
calendar.add(Calendar.DATE, 1);
startTime = calendar.getTime();
} return tables;
}
}

与Hint分片算法对应的Java查询方法 settlementMapper.selectByExample(example):

    public List<Settlement> querySettlements(SettlementExample example, String startTime, String endTime) {
// 组织查询时间,传入org.cellphone.finance.repo.sharding.HintTableShardingAlgorithm分片算法中以确认具体分表
String queryTime = startTime + CommonConst.UNDERLINE + endTime; // 获取HintManager
HintManager hintManager = HintManager.getInstance();
/*
* 添加数据源分片键值,使用Sharding Jdbc 3.x版本一定要添加数据源分片键值,否则无法使用HintTableShardingAlgorithm分片算法
* 若无分库,addDatabaseShardingValue方法的value字段随意填充
* 若有分库,addDatabaseShardingValue方法的value字段填充实际参数值
*/
hintManager.addDatabaseShardingValue("settlement_hint", StringUtils.EMPTY);
// 添加数据表分片键值
hintManager.addTableShardingValue("settlement_hint", queryTime);
List<Settlement> settlements = settlementMapper.selectByExample(example);
// 清除分片键值
hintManager.close();
return settlements;
}

以及该查询方法对应的SQL语句:

select * from settlement_hint t where t.pay_serial_number = ?

单元测试代码:

@Test
public void test003QuerySettlements() throws ParseException {
String startTime = "2018-04-03 00:00:00", endTime = "2018-04-05 00:00:00"; SettlementExample example = new SettlementExample();
SettlementExample.Criteria criteria = example.createCriteria();
criteria.andPaySerialNumberEqualTo(paySerialNumber); List<Settlement> settlements = repository.querySettlements(example, startTime, endTime); Assert.assertEquals("136********", settlements.get(0).getUserMobile());
}

三、源码分析

和2.0.3版本相比,3.1.0版本的路由入口变成了 io.shardingsphere.core.routing.type.standard.StandardRoutingEngine#route() ,但基本上区别不大,仅仅是多了一步需要确定路由的真实数据源,尽管数据源只有一个,也需要显式配置数据源路由算法。代码中标注了注释的部分都是路由代码比较核心的部分。

/*
* Copyright 2016-2018 shardingsphere.io.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* </p>
*/ package io.shardingsphere.core.routing.type.standard; import com.google.common.base.Optional;
import com.google.common.base.Preconditions;
import io.shardingsphere.api.algorithm.sharding.ShardingValue;
import io.shardingsphere.core.hint.HintManagerHolder;
import io.shardingsphere.core.optimizer.condition.ShardingCondition;
import io.shardingsphere.core.optimizer.condition.ShardingConditions;
import io.shardingsphere.core.optimizer.insert.InsertShardingCondition;
import io.shardingsphere.core.routing.strategy.ShardingStrategy;
import io.shardingsphere.core.routing.strategy.hint.HintShardingStrategy;
import io.shardingsphere.core.routing.type.RoutingEngine;
import io.shardingsphere.core.routing.type.RoutingResult;
import io.shardingsphere.core.routing.type.RoutingTable;
import io.shardingsphere.core.routing.type.TableUnit;
import io.shardingsphere.core.rule.BindingTableRule;
import io.shardingsphere.core.rule.DataNode;
import io.shardingsphere.core.rule.ShardingRule;
import io.shardingsphere.core.rule.TableRule;
import lombok.RequiredArgsConstructor; import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.LinkedHashSet;
import java.util.LinkedList;
import java.util.List; /**
* Standard routing engine.
*
* @author zhangliang
* @author maxiaoguang
* @author panjuan
*/
@RequiredArgsConstructor
public final class StandardRoutingEngine implements RoutingEngine { private final ShardingRule shardingRule; private final String logicTableName; private final ShardingConditions shardingConditions; /**
* 相比2.0.3版本,精简成了1行代码
*
* @return 路由结果
*/
@Override
public RoutingResult route() {
return generateRoutingResult(getDataNodes(shardingRule.getTableRuleByLogicTableName(logicTableName)));
} private RoutingResult generateRoutingResult(final Collection<DataNode> routedDataNodes) {
RoutingResult result = new RoutingResult();
for (DataNode each : routedDataNodes) {
TableUnit tableUnit = new TableUnit(each.getDataSourceName());
tableUnit.getRoutingTables().add(new RoutingTable(logicTableName, each.getTableName()));
result.getTableUnits().getTableUnits().add(tableUnit);
}
return result;
} /**
* 获取数据节点,即真实表
*
* @param tableRule 从XML读取的table rule
* @return 数据节点列表
*/
private Collection<DataNode> getDataNodes(final TableRule tableRule) {
// 判断是否是通过Hint分片策略进行路由
if (isRoutingByHint(tableRule)) {
return routeByHint(tableRule);
}
if (isRoutingByShardingConditions(tableRule)) {
return routeByShardingConditions(tableRule);
}
return routeByMixedConditions(tableRule);
} /**
* 判断是否是通过Hint分片策略进行路由
* <p>
* 数据源分片策略和数据表分片策略都必须是HintShardingStrategy,这意味者必须显式配置数据源Hint分片策略和数据表Hint分片策略
*
* @param tableRule 从XML读取的table rule
* @return
*/
private boolean isRoutingByHint(final TableRule tableRule) {
return shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy && shardingRule.getTableShardingStrategy(tableRule) instanceof HintShardingStrategy;
} /**
* 通过Hint分片策略进行路由
*
* @param tableRule
* @return
*/
private Collection<DataNode> routeByHint(final TableRule tableRule) {
return route(tableRule, getDatabaseShardingValuesFromHint(), getTableShardingValuesFromHint());
} private boolean isRoutingByShardingConditions(final TableRule tableRule) {
return !(shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy || shardingRule.getTableShardingStrategy(tableRule) instanceof HintShardingStrategy);
} private Collection<DataNode> routeByShardingConditions(final TableRule tableRule) {
return shardingConditions.getShardingConditions().isEmpty() ? route(tableRule, Collections.<ShardingValue>emptyList(), Collections.<ShardingValue>emptyList())
: routeByShardingConditionsWithCondition(tableRule);
} private Collection<DataNode> routeByShardingConditionsWithCondition(final TableRule tableRule) {
Collection<DataNode> result = new LinkedList<>();
for (ShardingCondition each : shardingConditions.getShardingConditions()) {
Collection<DataNode> dataNodes = route(tableRule, getShardingValuesFromShardingConditions(shardingRule.getDatabaseShardingStrategy(tableRule).getShardingColumns(), each),
getShardingValuesFromShardingConditions(shardingRule.getTableShardingStrategy(tableRule).getShardingColumns(), each));
reviseShardingConditions(each, dataNodes);
result.addAll(dataNodes);
}
return result;
} private Collection<DataNode> routeByMixedConditions(final TableRule tableRule) {
return shardingConditions.getShardingConditions().isEmpty() ? routeByMixedConditionsWithHint(tableRule) : routeByMixedConditionsWithCondition(tableRule);
} private Collection<DataNode> routeByMixedConditionsWithCondition(final TableRule tableRule) {
Collection<DataNode> result = new LinkedList<>();
for (ShardingCondition each : shardingConditions.getShardingConditions()) {
Collection<DataNode> dataNodes = route(tableRule, getDatabaseShardingValues(tableRule, each), getTableShardingValues(tableRule, each));
reviseShardingConditions(each, dataNodes);
result.addAll(dataNodes);
}
return result;
} private Collection<DataNode> routeByMixedConditionsWithHint(final TableRule tableRule) {
if (shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy) {
return route(tableRule, getDatabaseShardingValuesFromHint(), Collections.<ShardingValue>emptyList());
}
return route(tableRule, Collections.<ShardingValue>emptyList(), getTableShardingValuesFromHint());
} private List<ShardingValue> getDatabaseShardingValues(final TableRule tableRule, final ShardingCondition shardingCondition) {
ShardingStrategy dataBaseShardingStrategy = shardingRule.getDatabaseShardingStrategy(tableRule);
return isGettingShardingValuesFromHint(dataBaseShardingStrategy)
? getDatabaseShardingValuesFromHint() : getShardingValuesFromShardingConditions(dataBaseShardingStrategy.getShardingColumns(), shardingCondition);
} private List<ShardingValue> getTableShardingValues(final TableRule tableRule, final ShardingCondition shardingCondition) {
ShardingStrategy tableShardingStrategy = shardingRule.getTableShardingStrategy(tableRule);
return isGettingShardingValuesFromHint(tableShardingStrategy)
? getTableShardingValuesFromHint() : getShardingValuesFromShardingConditions(tableShardingStrategy.getShardingColumns(), shardingCondition);
} private boolean isGettingShardingValuesFromHint(final ShardingStrategy shardingStrategy) {
return shardingStrategy instanceof HintShardingStrategy;
} /**
* 从HintManagerHolder中获取数据源分片值
*
* @return 数据源分片值列表
*/
private List<ShardingValue> getDatabaseShardingValuesFromHint() {
// getDatabaseShardingValue方法实现有点恶心,不兼容大小写...
Optional<ShardingValue> shardingValueOptional = HintManagerHolder.getDatabaseShardingValue(logicTableName);
return shardingValueOptional.isPresent() ? Collections.singletonList(shardingValueOptional.get()) : Collections.<ShardingValue>emptyList();
} private List<ShardingValue> getTableShardingValuesFromHint() {
// getTableShardingValue方法实现有点恶心,不兼容大小写...
Optional<ShardingValue> shardingValueOptional = HintManagerHolder.getTableShardingValue(logicTableName);
return shardingValueOptional.isPresent() ? Collections.singletonList(shardingValueOptional.get()) : Collections.<ShardingValue>emptyList();
} private List<ShardingValue> getShardingValuesFromShardingConditions(final Collection<String> shardingColumns, final ShardingCondition shardingCondition) {
List<ShardingValue> result = new ArrayList<>(shardingColumns.size());
for (ShardingValue each : shardingCondition.getShardingValues()) {
Optional<BindingTableRule> bindingTableRule = shardingRule.findBindingTableRule(logicTableName);
if ((logicTableName.equals(each.getLogicTableName()) || bindingTableRule.isPresent() && bindingTableRule.get().hasLogicTable(logicTableName))
&& shardingColumns.contains(each.getColumnName())) {
result.add(each);
}
}
return result;
} /**
* 路由,获取真实表列表
*
* @param tableRule 从XML读取的table rule
* @param databaseShardingValues 数据源分片值
* @param tableShardingValues 数据表分片值
* @return 真实表列表
*/
private Collection<DataNode> route(final TableRule tableRule, final List<ShardingValue> databaseShardingValues, final List<ShardingValue> tableShardingValues) {
Collection<String> routedDataSources = routeDataSources(tableRule, databaseShardingValues);
Collection<DataNode> result = new LinkedList<>();
for (String each : routedDataSources) {
result.addAll(routeTables(tableRule, each, tableShardingValues));
}
return result;
} /**
* 路由到真实数据源
*
* @param tableRule 从XML读取的table rule
* @param databaseShardingValues 数据源分片值
* @return 真实数据源列表
*/
private Collection<String> routeDataSources(final TableRule tableRule, final List<ShardingValue> databaseShardingValues) {
Collection<String> availableTargetDatabases = tableRule.getActualDatasourceNames();
if (databaseShardingValues.isEmpty()) {
return availableTargetDatabases;
}
Collection<String> result = new LinkedHashSet<>(shardingRule.getDatabaseShardingStrategy(tableRule).doSharding(availableTargetDatabases, databaseShardingValues));
Preconditions.checkState(!result.isEmpty(), "no database route info");
return result;
} /**
* 路由到真实数据表,和2.0.3版本没啥区别
*
* @param tableRule 从XML读取的table rule
* @param routedDataSource 已确认好的数据源
* @param tableShardingValues 数据表分片值
* @return 真实表列表
*/
private Collection<DataNode> routeTables(final TableRule tableRule, final String routedDataSource, final List<ShardingValue> tableShardingValues) {
Collection<String> availableTargetTables = tableRule.getActualTableNames(routedDataSource);
Collection<String> routedTables = new LinkedHashSet<>(tableShardingValues.isEmpty() ? availableTargetTables
: shardingRule.getTableShardingStrategy(tableRule).doSharding(availableTargetTables, tableShardingValues));
Preconditions.checkState(!routedTables.isEmpty(), "no table route info");
Collection<DataNode> result = new LinkedList<>();
for (String each : routedTables) {
result.add(new DataNode(routedDataSource, each));
}
return result;
} private void reviseShardingConditions(final ShardingCondition each, final Collection<DataNode> dataNodes) {
if (each instanceof InsertShardingCondition) {
((InsertShardingCondition) each).getDataNodes().addAll(dataNodes);
}
}
}

分析到此,Sharding-JDBC 3.1.0版本可以支持分片键不存在于SQL中和数据表结构中的使用场景。但3.1.0版本还有一个比较恶心的地方,Sharding-JDBC在初始化时,会连接数据库获取数据表的元数据,包括需要水平切分的表和不需水平切分的表。代码如下:

/*
* Copyright 2016-2018 shardingsphere.io.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* </p>
*/ package io.shardingsphere.core.metadata.table.executor; import com.google.common.base.Optional;
import io.shardingsphere.core.exception.ShardingException;
import io.shardingsphere.core.executor.ShardingExecuteEngine;
import io.shardingsphere.core.metadata.datasource.DataSourceMetaData;
import io.shardingsphere.core.metadata.datasource.ShardingDataSourceMetaData;
import io.shardingsphere.core.metadata.table.TableMetaData;
import io.shardingsphere.core.rule.ShardingRule;
import io.shardingsphere.core.rule.TableRule; import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.Collection;
import java.util.HashMap;
import java.util.LinkedHashSet;
import java.util.Map; /**
* Table meta data initializer.
*
* @author zhangliang
*/
public final class TableMetaDataInitializer { private final ShardingDataSourceMetaData shardingDataSourceMetaData; private final TableMetaDataConnectionManager connectionManager; private final TableMetaDataLoader tableMetaDataLoader; public TableMetaDataInitializer(final ShardingDataSourceMetaData shardingDataSourceMetaData, final ShardingExecuteEngine executeEngine,
final TableMetaDataConnectionManager connectionManager, final int maxConnectionsSizePerQuery, final boolean isCheckingMetaData) {
this.shardingDataSourceMetaData = shardingDataSourceMetaData;
this.connectionManager = connectionManager;
tableMetaDataLoader = new TableMetaDataLoader(shardingDataSourceMetaData, executeEngine, connectionManager, maxConnectionsSizePerQuery, isCheckingMetaData);
} /**
* Load all table meta data.
*
* @param shardingRule sharding rule
* @return all table meta data
*/
public Map<String, TableMetaData> load(final ShardingRule shardingRule) {
Map<String, TableMetaData> result = new HashMap<>();
try {
// 加载需要水平切分的表元数据
result.putAll(loadShardingTables(shardingRule));
// 加载不需水平切分的表元数据,如果数据库中表数量很大,这里耗时很久...
result.putAll(loadDefaultTables(shardingRule));
} catch (final SQLException ex) {
throw new ShardingException(ex);
}
return result;
} private Map<String, TableMetaData> loadShardingTables(final ShardingRule shardingRule) throws SQLException {
Map<String, TableMetaData> result = new HashMap<>(shardingRule.getTableRules().size(), 1);
for (TableRule each : shardingRule.getTableRules()) {
result.put(each.getLogicTable(), tableMetaDataLoader.load(each.getLogicTable(), shardingRule));
}
return result;
} private Map<String, TableMetaData> loadDefaultTables(final ShardingRule shardingRule) throws SQLException {
Map<String, TableMetaData> result = new HashMap<>(shardingRule.getTableRules().size(), 1);
Optional<String> actualDefaultDataSourceName = shardingRule.findActualDefaultDataSourceName();
if (actualDefaultDataSourceName.isPresent()) {
for (String each : getAllTableNames(actualDefaultDataSourceName.get())) {
result.put(each, tableMetaDataLoader.load(each, shardingRule));
}
}
return result;
} /**
* 连接数据库,获取所有表元数据
*
* @param dataSourceName 数据源名称
* @return 所有数据表元数据列表
* @throws SQLException
*/
private Collection<String> getAllTableNames(final String dataSourceName) throws SQLException {
Collection<String> result = new LinkedHashSet<>();
DataSourceMetaData dataSourceMetaData = shardingDataSourceMetaData.getActualDataSourceMetaData(dataSourceName);
String catalog = null == dataSourceMetaData ? null : dataSourceMetaData.getSchemeName();
try (Connection connection = connectionManager.getConnection(dataSourceName);
ResultSet resultSet = connection.getMetaData().getTables(catalog, null, null, new String[]{"TABLE"})) {
while (resultSet.next()) {
String tableName = resultSet.getString("TABLE_NAME");
if (!tableName.contains("$")) {
result.add(tableName);
}
}
}
return result;
}
}

Sharding-JDBC(三)3.1.0版本实践的更多相关文章

  1. sharding jdbc(sphere) 3.1.0 spring boot配置

    sharding jdbc 2.x系列详解参见https://www.cnblogs.com/zhjh256/p/9221634.html. 最近将sharding jdbc的配置从xml切换到了sp ...

  2. MySQL的JDBC驱动(8.0版本)

    1.引用外部库  mysql-connector-java-8.0.版本的jar 2.jdbc驱动类:com.mysql.jdbc.Driver  改成 com.mysql.cj.jdbc.Drive ...

  3. FlinkCDC 2.0使用实践体验

    一.背景说明 所谓CDC:全称是 Change Data Capture ,在广义的概念上,只要能捕获数据变更的技术,我们都可以称为 CDC .通常我们说的 CDC 技术主要面向数据库的变更,是一种用 ...

  4. Sharding-JDBC(二)2.0.3版本实践

    目录 一.Sharding-JDBC依赖 二.分片策略 1. 标准分片策略 2. 复合分片策略 3. Inline表达式分片策略 4. 通过Hint而非SQL解析的方式分片的策略 5. 不分片的策略 ...

  5. 使用8.0版本jdbc驱动连接数据库操作

    转自:https://blog.csdn.net/qq_41734885/article/details/80562815 首先,我居然不能用navicat客户端连接上mysql8.0数据库报1251 ...

  6. 三、主流区块链技术特点及Hyperledger Fabric V1.0版本特点

    一.Hyperledger fabric V1.0 架构 1.逻辑架构: 2.区块链网络 3.运行时架构 二.架构总结 1.架构要点 分拆Peer的功能,将Blockchain的数据维护和共识服务进行 ...

  7. ElasticSearch5.0+版本分词热更新实践记录

    前言 刚开始接触ElasticSearch的时候,版本才是2.3.4,短短的时间,现在都更新到5.0+版本了.分词和head插件好像用法也不一样了,本博客记录如何配置Elasticsearch的Hea ...

  8. mysql 数据库8.0版本,jdbc驱动连接问题

    前言 8.0版本的mysql数据的连接 与 5.0的有所不同,下面直接贴出  8.0版本应该有的 jdbc驱动连接,还有 mysql 的jdbc jar包要8.0以上的 内容如下 : jdbc.dri ...

  9. Sharding jdbc 强制路由策略(HintShardingStrategy)使用记录

    背景 随着项目运行时间逐渐增加,数据库中的数据也越来越多,虽然加索引,优化查询,但是数据量太大,还是会影响查询效率,也给数据库增加了负载. 再加上冷数据基本不使用的场景,决定采用分表来处理数据,从而来 ...

随机推荐

  1. 【VS开发】VS2015没修改源文件也导致重新编译的解决办法

    在使用VS2010编译C++程序的时候,每次修改工程中的某一个文件,点击"生成-仅用于项目-仅生成**"时,往往都是整个工程都需要重新编译一遍.由于这个工程代码量太大,每次编译完成 ...

  2. Python 内置函数--super()

    描述 super() 函数是用于调用父类(超类)的一个方法. super 是用来解决多重继承问题的,直接用类名调用父类方法在使用单继承的时候没问题,但是如果使用多继承,会涉及到查找顺序(MRO).重复 ...

  3. Knockout.js之初印象

    最近在学Knockout.js,想要把看到的东西用blog记录下来. Knockout.js是一个MVVM的框架,突然想起之前有人问我哪些是MVVM框架?我回答了angular.js和bootstra ...

  4. qmake, makefile, make是什么东东,makefile简介!

    qmake是一个协助简化跨平台开发的构建过程 的工具,Qt附带的工具之一 .qmake能够自动生成Makefile.Microsoft Visual Studio 专案文件 和 xcode 专案文件. ...

  5. ERP通过JAVA流的形式将数据传到外围系统

    1.ERP封装数据成XML写入数据库服务器指定文件 --指定相关文件信息 v_file_path := '/u01/test/app/fs1/EBSapps/appl/cux/12.0.0/forms ...

  6. Python脚本-自动下载安装

    #coding=utf-8 import os import sys if os.getuid() == 0: pass else: print 'no' sys.exit(1) version = ...

  7. AX 2012 model应用

    说句实话,AX2012 model真心不好用,当你开发好,把Model到入到客户环境时, 要防止有冲突,假如出现冲突的话,还必须去删除另外一个Model里面冲突的代码,真心麻烦. 下面给一个ax 导入 ...

  8. Java对象深拷贝浅拷贝总结

    目录 深拷贝 1. 手动new 2. clone方法 3. java自带序列化 4. json序列化 性能测试 深拷贝总结 浅拷贝 1. spring BeanUtils(Apache BeanUti ...

  9. Linux中添加用户与删除用户

    注意:添加用户和删除用户需要root来执行. 添加用户 用useradd命令,例如: # useradd -d/home/tom -s/bin/bash -u1000 tom  这样就添加了新用户to ...

  10. c#Queue队列的使用

    消息队列 队列(System.Collections.Queue)代表了一个先进先出的对象集合.当您需要对各项进行先进先出的访问时,则使用队列.当您在列表中添加一项,称为入队,当您从列表中移除一项时, ...