一、水平分割

1、水平分庫
1)、概念：
以字段為依據，按照一定策略，將一個庫中的數據拆分到多個庫中。
2)、結果
每個庫的結構都一樣；數據都不一樣；
所有庫的并集是全量數據；
2、水平分表
1)、概念
以字段為依據，按照一定策略，將一個表中的數據拆分到多個表中。
2)、結果
每個表的結構都一樣；數據都不一樣；
所有表的并集是全量數據；

二、Shard-jdbc 中間件

1、架構圖

2、特點
1)、Sharding-JDBC直接封裝JDBC API，舊代碼遷移成本幾乎為零。
2)、適用于任何基于Java的ORM框架，如Hibernate、Mybatis等 。
3)、可基于任何第三方的數據庫連接池，如DBCP、C3P0、 BoneCP、Druid等。
4)、以jar包形式提供服務，無proxy代理層，無需額外部署，無其他依賴。
5)、分片策略靈活，可支持等號、between、in等多維度分片，也可支持多分片鍵。
6)、SQL解析功能完善，支持聚合、分組、排序、limit、or等查詢。

三、項目演示

1、項目結構

springboot 2.0 版本 druid 1.1.13 版本 sharding-jdbc 3.1 版本

2、數據庫配置

一臺基礎庫映射（shard_one） 兩臺庫做分庫分表（shard_two，shard_three）。 表使用：table_one，table_two

3、核心代碼塊

數據源配置文件

spring:datasource:# 數據源：shard_onedataOne:type: com.alibaba.druid.pool.DruidDataSourcedruid:driverClassName: com.mysql.jdbc.Driverurl: jdbc:mysql://localhost:3306/shard_one?useUnicode=true&characterEncoding=UTF8&zeroDateTimeBehavior=convertToNull&useSSL=falseusername: rootpassword: 123initial-size: 10max-active: 100min-idle: 10max-wait: 60000pool-prepared-statements: truemax-pool-prepared-statement-per-connection-size: 20time-between-eviction-runs-millis: 60000min-evictable-idle-time-millis: 300000max-evictable-idle-time-millis: 60000validation-query: SELECT 1 FROM DUAL# validation-query-timeout: 5000test-on-borrow: falsetest-on-return: falsetest-while-idle: trueconnectionProperties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000# 數據源：shard_twodataTwo:type: com.alibaba.druid.pool.DruidDataSourcedruid:driverClassName: com.mysql.jdbc.Driverurl: jdbc:mysql://localhost:3306/shard_two?useUnicode=true&characterEncoding=UTF8&zeroDateTimeBehavior=convertToNull&useSSL=falseusername: rootpassword: 123initial-size: 10max-active: 100min-idle: 10max-wait: 60000pool-prepared-statements: truemax-pool-prepared-statement-per-connection-size: 20time-between-eviction-runs-millis: 60000min-evictable-idle-time-millis: 300000max-evictable-idle-time-millis: 60000validation-query: SELECT 1 FROM DUAL# validation-query-timeout: 5000test-on-borrow: falsetest-on-return: falsetest-while-idle: trueconnectionProperties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000# 數據源：shard_threedataThree:type: com.alibaba.druid.pool.DruidDataSourcedruid:driverClassName: com.mysql.jdbc.Driverurl: jdbc:mysql://localhost:3306/shard_three?useUnicode=true&characterEncoding=UTF8&zeroDateTimeBehavior=convertToNull&useSSL=falseusername: rootpassword: 123initial-size: 10max-active: 100min-idle: 10max-wait: 60000pool-prepared-statements: truemax-pool-prepared-statement-per-connection-size: 20time-between-eviction-runs-millis: 60000min-evictable-idle-time-millis: 300000max-evictable-idle-time-millis: 60000validation-query: SELECT 1 FROM DUAL# validation-query-timeout: 5000test-on-borrow: falsetest-on-return: falsetest-while-idle: trueconnectionProperties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000

數據庫分庫策略

/*** 數據庫映射計算*/ public class DataSourceAlg implements PreciseShardingAlgorithm<String> {private static Logger LOG = LoggerFactory.getLogger(DataSourceAlg.class);@Overridepublic String doSharding(Collection<String> names, PreciseShardingValue<String> value) {LOG.debug("分庫算法參數 {},{}",names,value);int hash = HashUtil.rsHash(String.valueOf(value.getValue()));return "ds_" + ((hash % 2) + 2) ;} }

數據表1分表策略

/*** 分表算法*/ public class TableOneAlg implements PreciseShardingAlgorithm<String> {private static Logger LOG = LoggerFactory.getLogger(TableOneAlg.class);/*** 該表每個庫分5張表*/@Overridepublic String doSharding(Collection<String> names, PreciseShardingValue<String> value) {LOG.debug("分表算法參數 {},{}",names,value);int hash = HashUtil.rsHash(String.valueOf(value.getValue()));return "table_one_" + (hash % 5+1);} }

數據表2分表策略

/*** 分表算法*/ public class TableTwoAlg implements PreciseShardingAlgorithm<String> {private static Logger LOG = LoggerFactory.getLogger(TableTwoAlg.class);/*** 該表每個庫分5張表*/@Overridepublic String doSharding(Collection<String> names, PreciseShardingValue<String> value) {LOG.debug("分表算法參數 {},{}",names,value);int hash = HashUtil.rsHash(String.valueOf(value.getValue()));return "table_two_" + (hash % 5+1);} }

數據源集成配置

/*** 數據庫分庫分表配置*/ @Configuration public class ShardJdbcConfig {// 省略了 druid 配置，源碼中有/*** Shard-JDBC 分庫配置*/@Beanpublic DataSource dataSource (@Autowired DruidDataSource dataOneSource,@Autowired DruidDataSource dataTwoSource,@Autowired DruidDataSource dataThreeSource) throws Exception {ShardingRuleConfiguration shardJdbcConfig = new ShardingRuleConfiguration();shardJdbcConfig.getTableRuleConfigs().add(getTableRule01());shardJdbcConfig.getTableRuleConfigs().add(getTableRule02());shardJdbcConfig.setDefaultDataSourceName("ds_0");Map<String,DataSource> dataMap = new LinkedHashMap<>() ;dataMap.put("ds_0",dataOneSource) ;dataMap.put("ds_2",dataTwoSource) ;dataMap.put("ds_3",dataThreeSource) ;Properties prop = new Properties();return ShardingDataSourceFactory.createDataSource(dataMap, shardJdbcConfig, new HashMap<>(), prop);}/*** Shard-JDBC 分表配置*/private static TableRuleConfiguration getTableRule01() {TableRuleConfiguration result = new TableRuleConfiguration();result.setLogicTable("table_one");result.setActualDataNodes("ds_${2..3}.table_one_${1..5}");result.setDatabaseShardingStrategyConfig(new StandardShardingStrategyConfiguration("phone", new DataSourceAlg()));result.setTableShardingStrategyConfig(new StandardShardingStrategyConfiguration("phone", new TableOneAlg()));return result;}private static TableRuleConfiguration getTableRule02() {TableRuleConfiguration result = new TableRuleConfiguration();result.setLogicTable("table_two");result.setActualDataNodes("ds_${2..3}.table_two_${1..5}");result.setDatabaseShardingStrategyConfig(new StandardShardingStrategyConfiguration("phone", new DataSourceAlg()));result.setTableShardingStrategyConfig(new StandardShardingStrategyConfiguration("phone", new TableTwoAlg()));return result;} }

測試代碼執行流程

@RestController public class ShardController {@Resourceprivate ShardService shardService ;/*** 1、建表流程*/@RequestMapping("/createTable")public String createTable (){shardService.createTable();return "success" ;}/*** 2、生成表 table_one 數據*/@RequestMapping("/insertOne")public String insertOne (){shardService.insertOne();return "SUCCESS" ;}/*** 3、生成表 table_two 數據*/@RequestMapping("/insertTwo")public String insertTwo (){shardService.insertTwo();return "SUCCESS" ;}/*** 4、查詢表 table_one 數據*/@RequestMapping("/selectOneByPhone/{phone}")public TableOne selectOneByPhone (@PathVariable("phone") String phone){return shardService.selectOneByPhone(phone);}/*** 5、查詢表 table_one 數據*/@RequestMapping("/selectTwoByPhone/{phone}")public TableTwo selectTwoByPhone (@PathVariable("phone") String phone){return shardService.selectTwoByPhone(phone);} }

四、項目源碼

總結

以上是生活随笔為你收集整理的架构组件：基于shard-jdbc中间件，实现数据分库分表的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。