mongodb 3.4 学习 (四）分片
https://www.linode.com/docs/databases/mongodb/build-database-clusters-with-mongodb
由三部分组成
shard: 每个shard包括一个所有shard数据的子集. 每个shard能够部署成replica set
mongos: 做为查询路由, 提供clientt与sharded cluster的接口
config servers: 存储群集的metadata和配置，必须部署为replica set
# 生产环境部署
kvm-70-102 10.0.70.102 # mongos
kvm-70-103 10.0.70.103 # config
kvm-70-102 10.0.70.104 # shard1
kvm-70-103 10.0.70.105 # shard2
config
# 建立数据目录
mkdir -p /opt/mongo_{27021,27022,27023}
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
storage:
dbPath: /opt/mongo_27021
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: config
sharding:
clusterRole: configsvr
EOF
# 配置文件
cat > /etc/mongod_27022.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27022.log
storage:
dbPath: /opt/mongo_27022
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27022.pid
net:
port: 27022
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: config
sharding:
clusterRole: configsvr
EOF
# 配置文件
cat > /etc/mongod_27023.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27023.log
storage:
dbPath: /opt/mongo_27023
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27023.pid
net:
port: 27023
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: config
sharding:
clusterRole: configsvr
EOF
# 启动复制集
mongod --config /etc/mongod_27021.conf
mongod --config /etc/mongod_27022.conf
mongod --config /etc/mongod_27023.conf
# 配置config
mongo --port 27021
rs.initiate(
{
_id: 'config',
configsvr: true,
members: [
{ _id : 0, host : '10.0.70.103:27021' },
{ _id : 1, host : '10.0.70.103:27022' },
{ _id : 2, host : '10.0.70.103:27023' }
]
}
)
{ "ok" : 1 }
shard1
# 建立数据目录
mkdir -p /opt/mongo_{27021,27022,27023}
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
storage:
dbPath: /opt/mongo_27021
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27022.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27022.log
storage:
dbPath: /opt/mongo_27022
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27022.pid
net:
port: 27022
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27023.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27023.log
storage:
dbPath: /opt/mongo_27023
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27023.pid
net:
port: 27023
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
# 启动复制集
mongod --config /etc/mongod_27021.conf
mongod --config /etc/mongod_27022.conf
mongod --config /etc/mongod_27023.conf
# 配置config
mongo --port 27021
rs.initiate(
{
_id: 'shard1',
members: [
{ _id : 0, host : '10.0.70.104:27021' },
{ _id : 1, host : '10.0.70.104:27022' },
{ _id : 2, host : '10.0.70.104:27023' }
]
}
)
{ "ok" : 1 }
shard2
# 建立数据目录
mkdir -p /opt/mongo_{27021,27022,27023}
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
storage:
dbPath: /opt/mongo_27021
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27022.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27022.log
storage:
dbPath: /opt/mongo_27022
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27022.pid
net:
port: 27022
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27023.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27023.log
storage:
dbPath: /opt/mongo_27023
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27023.pid
net:
port: 27023
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
# 启动复制集
mongod --config /etc/mongod_27021.conf
mongod --config /etc/mongod_27022.conf
mongod --config /etc/mongod_27023.conf
# 配置config
mongo --port 27021
rs.initiate(
{
_id: 'shard2',
members: [
{ _id : 0, host : '10.0.70.105:27021' },
{ _id : 1, host : '10.0.70.105:27022' },
{ _id : 2, host : '10.0.70.105:27023' }
]
}
)
{ "ok" : 1 }
mongos
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
sharding:
configDB: config/10.0.70.103:27021,10.0.70.103:27022,10.0.70.103:27023
EOF
# 启动mongos
mongos --config /etc/mongod_27021.conf
# 配置
mongo --port 27021
# 添加分片
sh.addShard('shard1/10.0.70.104:27021')
sh.addShard('shard1/10.0.70.104:27022')
sh.addShard('shard1/10.0.70.104:27023')
sh.addShard('shard2/10.0.70.105:27021')
sh.addShard('shard2/10.0.70.105:27022')
sh.addShard('shard2/10.0.70.105:27023')
# 建立超级用户
use admin
db.createUser(
{
user: 'admin',
pwd: '@admin',
roles: [ { role: 'root', db: 'admin' } ]
}
);
# 认证登录
db.auth('admin', '@admin')
# app数据库demo表插入100000条数据
use app
for (var i = 1; i < 100000; i++) db.demo.insert({id: i, name: 'ken'})
WriteResult({ "nInserted" : 1 })
# 查看分片状态
sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("59261bca37ceff575b36ef09")
}
shards:
{ "_id" : "shard1", "host" : "shard1/10.0.70.104:27021,10.0.70.104:27022,10.0.70.104:27023", "state" : 1 }
{ "_id" : "shard2", "host" : "shard2/10.0.70.105:27021,10.0.70.105:27022,10.0.70.105:27023", "state" : 1 }
active mongoses:
"3.4.4" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Balancer lock taken at Thu May 25 2017 07:48:42 GMT+0800 (CST) by ConfigServer:Balancer
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "app", "primary" : "shard1", "partitioned" : false }
# 结果显示没有分区，记录插在shard1服务器
# 配置分片
# 允许分片的数据库
sh.enableSharding('app')
# 为用做shard key的字段建立索引，实际中可以用uuid字段
use app
db.demo.createIndex({id: 1})
{
"raw" : {
"shard1/10.0.70.104:27021,10.0.70.104:27022,10.0.70.104:27023" : {
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"ok" : 1,
"$gleStats" : {
"lastOpTime" : {
"ts" : Timestamp(1495675335, 1),
"t" : NumberLong(1)
},
"electionId" : ObjectId("7fffffff0000000000000001")
}
}
},
"ok" : 1
}
# 允许分片的collection，指定shard key的字段
sh.shardCollection('app.demo', { id : 'hashed' })
{ "collectionsharded" : "app.demo", "ok" : 1 }
# 插入数据
for (var i = 1; i < 10000; i++) db.demo.insert({id: i, name: 'ken'})
WriteResult({ "nInserted" : 1 })
# 查看状态，可以看到db与collection的分片状态
sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("59261bca37ceff575b36ef09")
}
shards:
{ "_id" : "shard1", "host" : "shard1/10.0.70.104:27021,10.0.70.104:27022,10.0.70.104:27023", "state" : 1 }
{ "_id" : "shard2", "host" : "shard2/10.0.70.105:27021,10.0.70.105:27022,10.0.70.105:27023", "state" : 1 }
active mongoses:
"3.4.4" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Balancer lock taken at Thu May 25 2017 07:48:42 GMT+0800 (CST) by ConfigServer:Balancer
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
1 : Success
databases:
{ "_id" : "app", "primary" : "shard1", "partitioned" : true }
app.demo
shard key: { "id" : "hashed" }
unique: false
balancing: true
chunks:
shard1 2
shard2 2
{ "id" : { "$minKey" : 1 } } -->> { "id" : NumberLong("-4611686018427387902") } on : shard1 Timestamp(2, 2)
{ "id" : NumberLong("-4611686018427387902") } -->> { "id" : NumberLong(0) } on : shard1 Timestamp(2, 3)
{ "id" : NumberLong(0) } -->> { "id" : NumberLong("4611686018427387902") } on : shard2 Timestamp(2, 4)
{ "id" : NumberLong("4611686018427387902") } -->> { "id" : { "$maxKey" : 1 } } on : shard2 Timestamp(2, 5)
# shard1
db.demo.find().count()
4992
# shard2
db.demo.find().count()
5007
如果在允许colletion前demo有数据，执行以下命令时会报错。
sh.shardCollection('app.demo', { id : 'hashed' })
{
"proposedKey" : {
"id" : "hashed"
},
"curIndexes" : [
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "app.demo"
},
{
"v" : 2,
"key" : {
"id" : 1
},
"name" : "id_1",
"ns" : "app.demo"
}
],
"ok" : 0,
"errmsg" : "please create an index that starts with the shard key before sharding."
}
做法是将collection删除，重建索引。
chunksize默认是64M，如果分片过慢的话，可以适当调大，如512M。mongos的配置文件里。
Hashed Sharding
Ranged Sharding
mongodb 3.4 学习 (四）分片的更多相关文章
- 搭建高可用mongodb集群（四）—— 分片
按照上一节中<搭建高可用mongodb集群(三)—— 深入副本集>搭建后还有两个问题没有解决: 从节点每个上面的数据都是对数据库全量拷贝,从节点压力会不会过大? 数据压力大到机器支撑不了的 ...
- 搭建高可用mongodb集群（四）—— 分片
按照上一节中<搭建高可用mongodb集群(三)-- 深入副本集>搭建后还有两个问题没有解决: 从节点每个上面的数据都是对数据库全量拷贝,从节点压力会不会过大? 数据压力大到机器支撑不了的 ...
- MongoDB的使用学习之（七）MongoDB的聚合查询（两种方式）附项目源码
先来张在路上…… 铛铛铛……项目源码下载地址:http://files.cnblogs.com/ontheroad_lee/MongoDBDemo.rar 此项目是用Maven创建的,没有使用Mave ...
- TweenMax动画库学习(四)
目录 TweenMax动画库学习(一) TweenMax动画库学习(二) TweenMax动画库学习(三) Tw ...
- SVG 学习<四> 基础API
目录 SVG 学习<一>基础图形及线段 SVG 学习<二>进阶 SVG世界,视野,视窗 stroke属性 svg分组 SVG 学习<三>渐变 SVG 学习<四 ...
- Android JNI学习(四)——JNI的常用方法的中文API
本系列文章如下: Android JNI(一)——NDK与JNI基础 Android JNI学习(二)——实战JNI之“hello world” Android JNI学习(三)——Java与Nati ...
- mongodb 搭建集群（分片+副本集）
mongodb 搭建集群(分片+副本集) 一.搭建结构图: 二.搭建步骤:
- SCARA——OpenGL入门学习四（颜色）
OpenGL入门学习[四] 本次学习的是颜色的选择.终于要走出黑白的世界了~~ OpenGL支持两种颜色模式:一种是RGBA,一种是颜色索引模式. 无论哪种颜色模式,计算机都必须为每一个像素保存一些数 ...
- mongodb常用命令学习笔记
mongodb常用命令学习笔记 创建数据库 use DATABASE_NAME eg: use users; 如果数据库不存在,则创建数据库,否则切换到指定数据库.要显示刚刚创建的数据库,需要向数据库 ...
随机推荐
- C++_新特性总结与未来的路
了解C++之后,可以阅读一些高级主题和面向对象编程相关的书籍: OOP有助于开发大型的项目,并提高其可靠性: OOP方法的基本活动之一就是发明能够模拟当前情况的类.当前情况被统称为问题域. 由于实际问 ...
- pip不是内部或外部命令也不是可运行的程序或批处理文件的问题
当我用windows电脑 pip install missingno 时 它居然会报pip不是内部或外部命令也不是可运行的程序或批处理文件的问题! 解决方法: 1)找到 pip.exe 所在位置,一般 ...
- 113th LeetCode Weekly Contest Reveal Cards In Increasing Order
In a deck of cards, every card has a unique integer. You can order the deck in any order you want. ...
- 解决页面使用overflow: scroll，overflow-y:hidden在iOS上滑动卡顿的问题
解决页面使用overflow: scroll,overflow-y:hidden在iOS上滑动卡顿的问题 div{ width: 100%; overflow-y: hidden; -webkit-o ...
- Home Web Server 1.9.1 build 164 - CGI Remote Code Execution复现
一. Home Web Server 1.9.1 build 164 - CGI Remote Code Execution复现 漏洞描述: Home Web Server允许调用CGI程序来通过P ...
- 解决windows10活动历史记录删除问题
Windows10日常办公过程中系统会记录很多活动历史记录信息,我是不希望我的活动历史记录随时可以被别人查看,虽然电脑有设置密码,但是办公电脑还是习惯将历史记录删除掉,并且永远禁用windows10的 ...
- Vue 项目启动抛出 Error/ No PostCSS Config found in
项目启动时抛出 Error: No PostCSS Config found in … 的错误表示某个 css 文件不能被引入 解决办法: module.exports = { plugins: { ...
- Oracle DG --检查
检查DG是否同步 01, 查看主备可切换状态 select switchover_status from v$database;select db_unique_name,open_mode,dat ...
- (转)python strip()函数 去空格\n\r\t函数的用法
原文:http://www.cnblogs.com/zdz8207/p/python_learn_note_20.html python3.4学习笔记(二十) python strip()函数 去空格 ...
- Zookeeper概念学习系列之zookeeper是什么？
1. Zookeeper是Hadoop的分布式协调服务. 2. 分布式应用程序可以基于它,来实现同步服务,配置维护和命名服务等. 3. zookeeper可以保证数据在zookeeper集群之间的数据 ...