mongodb 3.4 学习 (四）分片
https://www.linode.com/docs/databases/mongodb/build-database-clusters-with-mongodb
由三部分组成
shard: 每个shard包括一个所有shard数据的子集. 每个shard能够部署成replica set
mongos: 做为查询路由, 提供clientt与sharded cluster的接口
config servers: 存储群集的metadata和配置，必须部署为replica set
# 生产环境部署
kvm-70-102 10.0.70.102 # mongos
kvm-70-103 10.0.70.103 # config
kvm-70-102 10.0.70.104 # shard1
kvm-70-103 10.0.70.105 # shard2
config
# 建立数据目录
mkdir -p /opt/mongo_{27021,27022,27023}
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
storage:
dbPath: /opt/mongo_27021
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: config
sharding:
clusterRole: configsvr
EOF
# 配置文件
cat > /etc/mongod_27022.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27022.log
storage:
dbPath: /opt/mongo_27022
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27022.pid
net:
port: 27022
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: config
sharding:
clusterRole: configsvr
EOF
# 配置文件
cat > /etc/mongod_27023.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27023.log
storage:
dbPath: /opt/mongo_27023
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27023.pid
net:
port: 27023
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: config
sharding:
clusterRole: configsvr
EOF
# 启动复制集
mongod --config /etc/mongod_27021.conf
mongod --config /etc/mongod_27022.conf
mongod --config /etc/mongod_27023.conf
# 配置config
mongo --port 27021
rs.initiate(
{
_id: 'config',
configsvr: true,
members: [
{ _id : 0, host : '10.0.70.103:27021' },
{ _id : 1, host : '10.0.70.103:27022' },
{ _id : 2, host : '10.0.70.103:27023' }
]
}
)
{ "ok" : 1 }
shard1
# 建立数据目录
mkdir -p /opt/mongo_{27021,27022,27023}
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
storage:
dbPath: /opt/mongo_27021
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27022.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27022.log
storage:
dbPath: /opt/mongo_27022
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27022.pid
net:
port: 27022
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27023.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27023.log
storage:
dbPath: /opt/mongo_27023
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27023.pid
net:
port: 27023
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
# 启动复制集
mongod --config /etc/mongod_27021.conf
mongod --config /etc/mongod_27022.conf
mongod --config /etc/mongod_27023.conf
# 配置config
mongo --port 27021
rs.initiate(
{
_id: 'shard1',
members: [
{ _id : 0, host : '10.0.70.104:27021' },
{ _id : 1, host : '10.0.70.104:27022' },
{ _id : 2, host : '10.0.70.104:27023' }
]
}
)
{ "ok" : 1 }
shard2
# 建立数据目录
mkdir -p /opt/mongo_{27021,27022,27023}
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
storage:
dbPath: /opt/mongo_27021
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27022.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27022.log
storage:
dbPath: /opt/mongo_27022
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27022.pid
net:
port: 27022
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27023.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27023.log
storage:
dbPath: /opt/mongo_27023
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27023.pid
net:
port: 27023
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
# 启动复制集
mongod --config /etc/mongod_27021.conf
mongod --config /etc/mongod_27022.conf
mongod --config /etc/mongod_27023.conf
# 配置config
mongo --port 27021
rs.initiate(
{
_id: 'shard2',
members: [
{ _id : 0, host : '10.0.70.105:27021' },
{ _id : 1, host : '10.0.70.105:27022' },
{ _id : 2, host : '10.0.70.105:27023' }
]
}
)
{ "ok" : 1 }
mongos
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
sharding:
configDB: config/10.0.70.103:27021,10.0.70.103:27022,10.0.70.103:27023
EOF
# 启动mongos
mongos --config /etc/mongod_27021.conf
# 配置
mongo --port 27021
# 添加分片
sh.addShard('shard1/10.0.70.104:27021')
sh.addShard('shard1/10.0.70.104:27022')
sh.addShard('shard1/10.0.70.104:27023')
sh.addShard('shard2/10.0.70.105:27021')
sh.addShard('shard2/10.0.70.105:27022')
sh.addShard('shard2/10.0.70.105:27023')
# 建立超级用户
use admin
db.createUser(
{
user: 'admin',
pwd: '@admin',
roles: [ { role: 'root', db: 'admin' } ]
}
);
# 认证登录
db.auth('admin', '@admin')
# app数据库demo表插入100000条数据
use app
for (var i = 1; i < 100000; i++) db.demo.insert({id: i, name: 'ken'})
WriteResult({ "nInserted" : 1 })
# 查看分片状态
sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("59261bca37ceff575b36ef09")
}
shards:
{ "_id" : "shard1", "host" : "shard1/10.0.70.104:27021,10.0.70.104:27022,10.0.70.104:27023", "state" : 1 }
{ "_id" : "shard2", "host" : "shard2/10.0.70.105:27021,10.0.70.105:27022,10.0.70.105:27023", "state" : 1 }
active mongoses:
"3.4.4" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Balancer lock taken at Thu May 25 2017 07:48:42 GMT+0800 (CST) by ConfigServer:Balancer
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "app", "primary" : "shard1", "partitioned" : false }
# 结果显示没有分区，记录插在shard1服务器
# 配置分片
# 允许分片的数据库
sh.enableSharding('app')
# 为用做shard key的字段建立索引，实际中可以用uuid字段
use app
db.demo.createIndex({id: 1})
{
"raw" : {
"shard1/10.0.70.104:27021,10.0.70.104:27022,10.0.70.104:27023" : {
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"ok" : 1,
"$gleStats" : {
"lastOpTime" : {
"ts" : Timestamp(1495675335, 1),
"t" : NumberLong(1)
},
"electionId" : ObjectId("7fffffff0000000000000001")
}
}
},
"ok" : 1
}
# 允许分片的collection，指定shard key的字段
sh.shardCollection('app.demo', { id : 'hashed' })
{ "collectionsharded" : "app.demo", "ok" : 1 }
# 插入数据
for (var i = 1; i < 10000; i++) db.demo.insert({id: i, name: 'ken'})
WriteResult({ "nInserted" : 1 })
# 查看状态，可以看到db与collection的分片状态
sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("59261bca37ceff575b36ef09")
}
shards:
{ "_id" : "shard1", "host" : "shard1/10.0.70.104:27021,10.0.70.104:27022,10.0.70.104:27023", "state" : 1 }
{ "_id" : "shard2", "host" : "shard2/10.0.70.105:27021,10.0.70.105:27022,10.0.70.105:27023", "state" : 1 }
active mongoses:
"3.4.4" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Balancer lock taken at Thu May 25 2017 07:48:42 GMT+0800 (CST) by ConfigServer:Balancer
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
1 : Success
databases:
{ "_id" : "app", "primary" : "shard1", "partitioned" : true }
app.demo
shard key: { "id" : "hashed" }
unique: false
balancing: true
chunks:
shard1 2
shard2 2
{ "id" : { "$minKey" : 1 } } -->> { "id" : NumberLong("-4611686018427387902") } on : shard1 Timestamp(2, 2)
{ "id" : NumberLong("-4611686018427387902") } -->> { "id" : NumberLong(0) } on : shard1 Timestamp(2, 3)
{ "id" : NumberLong(0) } -->> { "id" : NumberLong("4611686018427387902") } on : shard2 Timestamp(2, 4)
{ "id" : NumberLong("4611686018427387902") } -->> { "id" : { "$maxKey" : 1 } } on : shard2 Timestamp(2, 5)
# shard1
db.demo.find().count()
4992
# shard2
db.demo.find().count()
5007
如果在允许colletion前demo有数据，执行以下命令时会报错。
sh.shardCollection('app.demo', { id : 'hashed' })
{
"proposedKey" : {
"id" : "hashed"
},
"curIndexes" : [
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "app.demo"
},
{
"v" : 2,
"key" : {
"id" : 1
},
"name" : "id_1",
"ns" : "app.demo"
}
],
"ok" : 0,
"errmsg" : "please create an index that starts with the shard key before sharding."
}
做法是将collection删除，重建索引。
chunksize默认是64M，如果分片过慢的话，可以适当调大，如512M。mongos的配置文件里。
Hashed Sharding
Ranged Sharding
mongodb 3.4 学习 (四）分片的更多相关文章
- 搭建高可用mongodb集群（四）—— 分片
按照上一节中<搭建高可用mongodb集群(三)—— 深入副本集>搭建后还有两个问题没有解决: 从节点每个上面的数据都是对数据库全量拷贝,从节点压力会不会过大? 数据压力大到机器支撑不了的 ...
- 搭建高可用mongodb集群（四）—— 分片
按照上一节中<搭建高可用mongodb集群(三)-- 深入副本集>搭建后还有两个问题没有解决: 从节点每个上面的数据都是对数据库全量拷贝,从节点压力会不会过大? 数据压力大到机器支撑不了的 ...
- MongoDB的使用学习之（七）MongoDB的聚合查询（两种方式）附项目源码
先来张在路上…… 铛铛铛……项目源码下载地址:http://files.cnblogs.com/ontheroad_lee/MongoDBDemo.rar 此项目是用Maven创建的,没有使用Mave ...
- TweenMax动画库学习(四)
目录 TweenMax动画库学习(一) TweenMax动画库学习(二) TweenMax动画库学习(三) Tw ...
- SVG 学习<四> 基础API
目录 SVG 学习<一>基础图形及线段 SVG 学习<二>进阶 SVG世界,视野,视窗 stroke属性 svg分组 SVG 学习<三>渐变 SVG 学习<四 ...
- Android JNI学习(四)——JNI的常用方法的中文API
本系列文章如下: Android JNI(一)——NDK与JNI基础 Android JNI学习(二)——实战JNI之“hello world” Android JNI学习(三)——Java与Nati ...
- mongodb 搭建集群（分片+副本集）
mongodb 搭建集群(分片+副本集) 一.搭建结构图: 二.搭建步骤:
- SCARA——OpenGL入门学习四（颜色）
OpenGL入门学习[四] 本次学习的是颜色的选择.终于要走出黑白的世界了~~ OpenGL支持两种颜色模式:一种是RGBA,一种是颜色索引模式. 无论哪种颜色模式,计算机都必须为每一个像素保存一些数 ...
- mongodb常用命令学习笔记
mongodb常用命令学习笔记 创建数据库 use DATABASE_NAME eg: use users; 如果数据库不存在,则创建数据库,否则切换到指定数据库.要显示刚刚创建的数据库,需要向数据库 ...
随机推荐
- 用 ASP.NET MVC 实现基于 XMLHttpRequest long polling（长轮询） 的 Comet
ASP.NET 计时器 http://www.cnblogs.com/dudu/archive/2011/10/17/2215321.html http://www.cnblogs.com/w ...
- Codeforces - 38G 可持久化Treap 区间操作
题意:\(n\)个人排队,每个人有重要度\(p\)和不要脸度\(c\),如果第\(i\)个人的重要度大于第\(i-1\)个人的重要度,那么他们之间可以交换,不要脸度-1,交换后先前的第\(i\)个人也 ...
- layer.open中content里面的元素追加click事件，触发不了
[注意] 事件要追加在触发弹出事件的点击事件里面 $('#feedback').click(function(){ layer.open({ content:'<div><div c ...
- Hibernate 4 Second Level Caching With EHCache
Hibernate 4 Second Level Caching With EHCache [From] http://www.codesenior.com/en/tutorial/Hibernate ...
- python--交互器，编译器
1在写python的时候调交互器的作用 唯一作用:调试代码 2.编译器
- Win10小娜关闭或删除进程
先来说下我为什么想尽方法关闭win10小娜:我觉得功能并不适用于我,即便不启用Cortana小娜,在Win10进程中也会看到Cortana小娜启动着,耗费了内存.CPU,而且主要的我的磁盘利用率等都居 ...
- PIXI 下落文字消除（3）
图片示例,简陋的图,记录下落过程, 1.创建应用实例并添加到DOM元素上. (会看到一个黑色画布,没有任何元素,接下来会在画布上创建文字) 2.创建 TextStyle 用来设置要显示字体样式 3. ...
- 【STM32学习笔记】STM32f407 使用4*4矩阵键盘
作者:李剀 出处:https://www.cnblogs.com/kevin-nancy/ 欢迎转载,但也请保留上面这段声明.谢谢! 写在前面: 这是本人第一次开始写博客,可能写的不是很好,也请大家谅 ...
- OpenGL Loading
什么是 OpenGL loading? OpenGL是一份API规范,并不是一个库.记住这点非常重要!它意味着每一个API背后的具体实现都依赖于你的GPU硬件.操作系统以及显卡驱动. OpenGL规范 ...
- php二维数组排序方法(array_multisort usort)
一维数组排序可以使用asort.ksort等一些方法进程排序,相对来说比较简单.二维数组的排序怎么实现呢?使用array_multisort和usort可以实现 例如像下面的数组: $users = ...