时间:2021-05-24
一、环境说明
1、我们prod环境MongoDB的集群架构是做的分片集群的部署,但是目前我们没有分片,即所有数据都在一个分片上,后期如果数量大,需要分配,集群随时可以分片,对业务方透明
2、各个角色的部署情况
角色 IP 端口 复制集名称 mongos 172.21.244.101,172.21.244.102,172.21.244.94 27000 无 config server 172.21.244.101,172.21.244.102,172.21.244.94 27100 repl_configsvr 存储节点(shard) 172.21.244.101,172.21.244.102,172.21.244.94 27101 shard1
3、MongoDB版本
mongos> db.version()4.0.4-62-g7e345a7二、基础信息准备
0、系统优化
echo "never" >/sys/kernel/mm/transparent_hugepage/enabledecho "never" >/sys/kernel/mm/transparent_hugepage/defrag1、下载MongoDB二进制文件
cd /chj/appwget ops.chehejia.com:9090/pkg/chj_mongodb_4.0.4.tar.gztar -zxvf chj_mongodb_4.0.4.tar.gz2、相关目录建立
#建立base目录mkdir /chj/data/mongodb/chj_db#把MongoDB二进制文件移动到base目录下的bin文件夹mv chj_mongodb_4.0.4/bin /chj/data/mongodb/chj_db/bin#建立认证文件目录mkdir /chj/data/mongodb/chj_db/auth#建立配置文件目录mkdir /chj/data/mongodb/chj_db/conf#建立config server的data和日志目录mkdir /chj/data/mongodb/chj_db/config/data -pmkdir /chj/data/mongodb/chj_db/config/log#建立mongos的日志目录mkdir /chj/data/mongodb/chj_db/mongos/log -p#建立数据节点data和日志目录 mkdir /chj/data/mongodb/chj_db/shard1/data -pmkdir /chj/data/mongodb/chj_db/shard1/log3、相关配置文件编写
A、mongos的配置文件编写
vim /chj/data/mongodb/chj_db/conf/mongos.confsystemLog: destination: file logAppend: true path: /chj/data/mongodb/chj_db/mongos/log/mongos.logprocessManagement: fork: true # fork and run in background pidFilePath: /chj/data/mongodb/chj_db/mongos/log/mongos.pid # location of pidfile timeZoneInfo: /usr/share/zoneinfonet: port: 27000 bindIpAll: true maxIncomingConnections: 1000 unixDomainSocket: enabled: true pathPrefix: /chj/data/mongodb/chj_db/mongos/log filePermissions: 0700security: keyFile: /chj/data/mongodb/chj_db/auth/keyfile.key# authorization: enabled#replication:sharding: configDB: repl_configsvr/172.21.244.101:27100,172.21.244.102:27100,172.21.244.94:27100B、config server的配置文件编写
vim /chj/data/mongodb/chj_db/conf/config.confsystemLog: destination: file logAppend: true path: /chj/data/mongodb/chj_db/config/log/congigsrv.logstorage: dbPath: /chj/data/mongodb/chj_db/config/data journal: enabled: true wiredTiger: engineConfig: directoryForIndexes: trueprocessManagement: fork: true # fork and run in background pidFilePath: /chj/data/mongodb/chj_db/config/log/configsrv.pid # location of pidfile timeZoneInfo: /usr/share/zoneinfonet: port: 27100 bindIp: 0.0.0.0 # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting. #bindIpAll: true maxIncomingConnections: 1000 unixDomainSocket: enabled: true pathPrefix: /chj/data/mongodb/chj_db/config/data filePermissions: 0700security: keyFile: /chj/data/mongodb/chj_db/auth/keyfile.key authorization: enabledreplication: replSetName: repl_configsvrsharding: clusterRole: configsvrC、存储节点的配置文件编写
vim /chj/data/mongodb/chj_db/conf/shard1.confsystemLog: destination: file logAppend: true path: /chj/data/mongodb/chj_db/shard1/log/shard1.logstorage: dbPath: /chj/data/mongodb/chj_db/shard1/data journal: enabled: true wiredTiger: engineConfig: directoryForIndexes: trueprocessManagement: fork: true # fork and run in background pidFilePath: /chj/data/mongodb/chj_db/shard1/log/shard1.pid # location of pidfile timeZoneInfo: /usr/share/zoneinfonet: port: 27101 bindIp: 0.0.0.0 # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting. #bindIpAll: true maxIncomingConnections: 1000 unixDomainSocket: enabled: true pathPrefix: /chj/data/mongodb/chj_db/shard1/data filePermissions: 0700security: keyFile: /chj/data/mongodb/chj_db/auth/keyfile.key authorization: enabledreplication: replSetName: shard1sharding: clusterRole: shardsvr4、生产key认证文件
echo "chj123456" >/chj/data/mongodb/chj_db/auth/keyfile.key#设置文件的权限为400,不然服务无法启动chmod 400 /chj/data/mongodb/chj_db/auth/keyfile.key三、集群初始化
1、启动 config server 服务
/chj/data/mongodb/chj_db/bin/mongod -f /chj/data/mongodb/chj_db/conf/config.conf2、初始化config server集群
#登录其中一个config server节点/chj/data/mongodb/chj_db/bin/mongo --port 27100#配置集群config = { _id:"repl_configsvr",members:[ {_id:0,host:"172.21.244.101:27100"}, {_id:1,host:"172.21.244.102:27100"}, {_id:2,host:"172.21.244.94:27100"}] }#初始化集群rs.initiate(config)PS:结果输出如下,说明集群初始化成功,可以通过rs.status()命令查看集群状态{ "ok" : 1, "$gleStats" : { "lastOpTime" : Timestamp(1557538260, 1), "electionId" : ObjectId("000000000000000000000000") }, "lastCommittedOpTime" : Timestamp(0, 0)}3、启动存储节点服务
/chj/data/mongodb/chj_db/bin/mongod -f /chj/data/mongodb/chj_db/conf/shard1.conf4、初始化存储集群
#登录你希望是主节点的服务器/chj/data/mongodb/chj_db/bin/mongo --port 27101#配置集群config = { _id:"shard1",members:[ {_id:0,host:"172.21.244.101:27101"}, {_id:1,host:"172.21.244.102:27101"},{_id:2,host:"172.21.244.94:27101",arbiterOnly:true}] }#初始化集群rs.initiate(config)PS:结果输出如下,说明集群初始化成功,可以通过rs.status()命令查看集群状态{ "ok" : 1 }5、添加存储集群的管理账号
登录主节点
6、启动mongos 服务
/chj/data/mongodb/chj_db/bin/mongos -f /chj/data/mongodb/chj_db/conf/mongos.conf7、添加config server的管理账号
登录任意一个mongos节点
8、把存储节点添加到mongos
登录任意一个mongos节点(如果是在上一步的窗口,需要退出重新登录)
四、交付业务方
1、建立应用账号
登录任意一个mongos节点/chj/data/mongodb/chj_db/bin/mongo --port 27000use admindb.auth('root','123456')#切到业务数据库use chj_db#建立读写账号db.createUser({ user: "chj_db_rw", pwd: "123456", roles: [ { role: "readWrite", db: "chj_db" }, { role: "dbOwner", db: "chj_db" } ]})#建立只读账号(根据业务需求确认是否需要)db.createUser({user: "chj_db_r",pwd: "123456",roles: [ { role: "read", db: "chj_db" } ]})2、交付开发人员信息
连接地址:172.21.244.101:27000,172.21.244.102:27000,172.21.244.94:27000库名:chj_db账号:chj_db_rw密码:123456五、数据库启用分片
如果后期业务量大,需要开启分片,配置如下
#指定需要分片的数据库mongos> sh.enableSharding("chj_db") { "ok" : 1, "operationTime" : Timestamp(1557546835, 3), "$clusterTime" : { "clusterTime" : Timestamp(1557546835, 3), "signature" : { "hash" : BinData(0,"bkrrr8Kxrr9j9udrDc/hURHld38="), "keyId" : NumberLong("6689575940508352541") } }}#在chj_db数据库和users集合中创建了name和age为升序的片键mongos> sh.shardCollection("chj_db.users",{name:1,age:1}) { "collectionsharded" : "chj_db.users", "collectionUUID" : UUID("59c0b99f-efff-4132-b489-f6c7e3d98f42"), "ok" : 1, "operationTime" : Timestamp(1557546861, 12), "$clusterTime" : { "clusterTime" : Timestamp(1557546861, 12), "signature" : { "hash" : BinData(0,"UBB1A/YODnmXwG5eAhgNLcKVzug="), "keyId" : NumberLong("6689575940508352541") } }}#查看分片情况mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5cd625e0da695346d740f749") } shards: { "_id" : "shard1", "host" : "shard1/172.21.244.101:27101,172.21.244.102:27101", "state" : 1 } active mongoses: "4.0.4-62-g7e345a7" : 3 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "chj_db", "primary" : "shard1", "partitioned" : true, "version" : { "uuid" : UUID("82088bc7-7b98-4033-843d-7058d8d959f6"), "lastMod" : 1 } } chj_db.users shard key: { "name" : 1, "age" : 1 } unique: false balancing: true chunks: shard1 1 { "name" : { "$minKey" : 1 }, "age" : { "$minKey" : 1 } } -->> { "name" : { "$maxKey" : 1 }, "age" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shard1 1 { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0)以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持。
声明:本页内容来源网络,仅供用户参考;我单位不保证亦不表示资料全面及准确无误,也不保证亦不表示这些资料为最新信息,如因任何原因,本网内容或者用户因倚赖本网内容造成任何损失或损害,我单位将不会负任何法律责任。如涉及版权问题,请提交至online#300.cn邮箱联系删除。
在mongodb里面存在另一种集群,就是分片技术,当数据量达到T级别的时候,我们的磁盘,内存就吃不消了,针对这样的场景我们该如何应对。一:分片mongodb采用
前言在MongoDB(版本3.2.9)中,分片集群(shardedcluster)是一种水平扩展数据库系统性能的方法,能够将数据集分布式存储在不同的分片(sha
在mongodb里面存在另一种集群,就是分片技术,跟sqlserver的表分区类似,我们知道当数据量达到T级别的时候,我们的磁盘,内存就吃不消了,针对这样的场景
01主分片是什么?分片集群中的每一个数据库都有一个主分片,这个主分片上保存了当前数据库中没有被分片的集合的数据,主分片(primaryshard)和主节点(pr
quartz集群能力:quartz集群分为水平集群和垂直集群,水平集群即将定时任务节点部署在不同的服务器,水平集群最大的问题就是时钟同步问题,quartz集群强