时下大数据时代,海量数据与吞吐量的数据库应用对单机的性能造成了较大的压力,将会发生CPU耗尽,存储压力大,可用资源耗尽等问题。
成都创新互联专注于企业成都全网营销、网站重做改版、株洲网站定制设计、自适应品牌网站建设、H5场景定制、商城网站建设、集团公司官网建设、成都外贸网站制作、高端网站制作、响应式网页设计等建站业务,价格优惠性价比高,为株洲等各大城市提供网站开发制作服务。
便出现了新的技术,分片技术。它是MongoDB用来将大型集合分割到不同服务器上所采用的的方法,它几乎是能够自动完成所有事情,只要告诉MongoDB要分配的数据,它就能够自动维护数据到不同服务器之间均衡存储。
Config server: 存储集群所有节点,分片数据路由信息,默认需要配置3个config server节点;
Mongos: 提供对外应用的访问,所有操作均通过mongos执行,一般有多个mongos节点,数据迁移和数据自动平衡;
Mongod: 存储应用数据记录,一般有多个Mongod节点,达到数据的分片目的。
1. 三台服务器配置如下:
IP地址 | 路由服务器 | 配置服务器 | shard1 | shard2 | shard3 |
---|---|---|---|---|---|
192.168.96.10 | 20000 | 21000 | 27001 | 27002 | 27003 |
192.168.96.11 | 20000 | 21000 | 27001 | 27002 | 27003 |
192.168.96.12 | 20000 | 21000 | 27001 | 27002 | 27003 |
2.操作系统:CentOS 7
3.关闭防火墙和Selinux功能
4.软件包:mongodb-linux-x86_64-rhel70-3.6.6.tgz 密码:4h77
tar -zxvf mongodb-linux-x86_64-rhel70-3.6.6.tgz -C /usr/local
cd /usr/local
mv mongodb-linux-x86_64-rhel70-3.6.6 mongodb
mkdir -p /data/mongodb/conf
mkdir -p /data/mongodb/mongos/log
mkdir -p /data/mongodb/config/data
mkdir -p /data/mongodb/config/log
#循环创建分片服务器目录
for i in 1 2 3
do
mkdir -p /data/mongodb/shard$i/data
mkdir -p /data/mongodb/shard$i/log
done
useradd -M -s /sbin/nologin mongo
chown -R mongo.mongo /usr/local/mongodb
chown -R mongo.mongo /data/mongodb
echo "PATH=/usr/local/mongodb/bin:$PATH" >> /etc/profile
source /etc/profile
vim ~/.bash_profile
ulimit -n 25000 //可以打开的大文件数量
ulimit -u 25000 //用户大可用的进程数
sysctl -w vm.zone_reclaim_mode=0 //当内存不足时,从其他节点分配内存
source ~/.bash_profile
vim /etc/rc.d/rc.local
......
//对大内存页面优化
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
chmod +x /etc/rc.d/rc.local
特别提醒:请检查三台服务器配置是否都已经配置并启动好
vim /data/mongodb/conf/config.conf
#配置文件内容
pidfilepath = /data/mongodb/config/log/config-srv.pid
dbpath = /data/mongodb/config/data
logpath = /data/mongodb/config/log/config-srv.log
logappend = true
bind_ip = 0.0.0.0
port = 21000
fork = true
#declare this is a config db of a cluster;
configsvr = true
#复制集名称
replSet=configs
#设置大连接数
maxConns=20000
vim /usr/lib/systemd/system/mongo-config.service
[Unit]
Description=mongodb-config
After=network.target remote-fs.target nss-lookup.target
[Service]
Type=forking
User=mongo
Group=mongo
ExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/config.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/config.conf
PrivateTmp=true
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable mongo-config
systemctl start mongo-config
特别提醒:三台服务器都配置完毕后,再进行以下初始化步骤
mongo --port 21000
//配置
> config={"_id":"configs",members:[{"_id":0,"host":"192.168.96.10:21000"},{"_id":1,"host":"192.168.96.11:21000"},{"_id":2,"host":"192.168.96.12:21000"}]}
//初始化
> rs.initiate(config)
//查看状态
> rs.status()
vim /data/mongodb/conf/shard1.conf
pidfilepath = /data/mongodb/shard1/log/shard1.pid
dbpath = /data/mongodb/shard1/data
logpath = /data/mongodb/shard1/log/shard1.log
logappend = true
journal = true
quiet = true
bind_ip = 0.0.0.0
port = 27001
fork = true
#复制集名称
replSet=shard1
#declare this is a shard db of a cluster;
shardsvr = true
#设置大连接数
maxConns=20000
vim /usr/lib/systemd/system/mongo-shard1.service
[Unit]
Description=mongodb-shard1
After= mongo-config.target network.target
[Service]
Type=forking
User=mongo
Group=mongo
ExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/shard1.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/shard1.conf
PrivateTmp=true
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable mongo-shard1
systemctl start mongo-shard1
特别提醒:三台服务器都配置完毕后,再进行以下初始化步骤
mongo --port 27001
> use admin
> config={
"_id":"shard1",members:[{"_id":0,"host":"192.168.96.10:27001",arbiterOnly:true},{"_id":1,"host":"192.168.96.11:27001"},{"_id":2,"host":"192.168.96.12:27001"}]}
> rs.initiate(config)
> rs.status()
vim /data/mongodb/conf/shard2.conf
pidfilepath = /data/mongodb/shard2/log/shard2.pid
dbpath = /data/mongodb/shard2/data
logpath = /data/mongodb/shard2/log/shard2.log
logappend = true
journal = true
quiet = true
bind_ip = 0.0.0.0
port = 27002
fork = true
#复制集名称
replSet=shard2
#declare this is a shard db of a cluster;
shardsvr = true
#设置大连接数
maxConns=20000
vim /usr/lib/systemd/system/mongo-shard2.service
[Unit]
Description=mongodb-shard2
After= mongo-config.target network.target
[Service]
Type=forking
User=mongo
Group=mongo
ExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/shard2.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/shard2.conf
PrivateTmp=true
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable mongo-shard2
systemctl start mongo-shard2
特别提醒:三台服务器都配置完毕后,再进行以下初始化步骤
mongo --port 27002
> use admin
> config={"_id":"shard2",members:[{ "_id":0,"host":"192.168.96.10:27002"}, {"_id":1,"host":"192.168.96.11:27002",arbiterOnly:true},{"_id":2,"host":"192.168.96.12:27002"}]}
> rs.initiate(config);
> rs.status()
vim /data/mongodb/conf/shard3.conf
pidfilepath = /data/mongodb/shard3/log/shard3.pid
dbpath = /data/mongodb/shard3/data
logpath = /data/mongodb/shard3/log/shard3.log
logappend = true
journal = true
quiet = true
bind_ip = 0.0.0.0
port = 27003
fork = true
#复制集名称
replSet=shard3
#declare this is a shard db of a cluster;
shardsvr = true
#设置大连接数
maxConns=20000
vim /usr/lib/systemd/system/mongo-shard3.service
```[Unit]
Description=mongodb-shard3
After= mongo-config.target network.target
[Service]
Type=forking
User=mongo
Group=mongo
ExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/shard3.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/shard3.conf
PrivateTmp=true
[Install]
WantedBy=multi-user.target
#### 3.创建启动脚本
systemctl daemon-reload
systemctl enable mongo-shard3
systemctl start mongo-shard3
**特别提醒:三台服务器都配置完毕后,再进行以下初始化步骤**
#### 4.初始化复制集(登录任意一台服务器进行初始化复制集)
> mongo --port 27003
use admin
config={"_id":"shard3",members:[{ "_id":0,"host":"192.168.96.10:27003"}, {"_id":1,"host":"192.168.96.11:27003"},{"_id":2,"host":"192.168.96.12:27003",arbiterOnly:true}]}
rs.initiate(config)
rs.status()
vim /data/mongodb/conf/mongos.conf
pidfilepath = /data/mongodb/mongos/log/mongos.pid
logpath = /data/mongodb/mongos/log/mongos.log
logappend = true
bind_ip = 0.0.0.0
port = 20000
fork = true
#监听的配置服务器的地址:端口,(重要、重要、重要)
configdb = configs/192.168.96.10:21000,192.168.96.11:21000,192.168.96.12:21000
#设置大连接数
maxConns=20000
vim /usr/lib/systemd/system/mongos.service
[Unit]
Description=Mongo Router Service
After=mongo-config.service
[Service]
Type=forking
User=mongo
Group=mongo
ExecStart=/usr/local/mongodb/bin/mongos --config /data/mongodb/conf/mongos.conf
Restart=on-failure
[Install]
WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable mongos
systemctl start mongos
特别提醒:三台服务器都配置完毕后,再进行以下步骤
mongo --port 20000
> use admin
> sh.addShard("shard1/192.168.96.10:27001,192.168.96.11:27001,192.168.96.12:27001")
sh.addShard("shard2/192.168.96.10:27002,192.168.96.11:27002,192.168.96.12:27002")
sh.addShard("shard3/192.168.96.10:27003,192.168.96.11:27003,192.168.96.12:27003")
> sh.status()
> use school
> for(i=1;i<=1000;i++){db.user.insert({"id":i,"name":"jack"+i})}
//开启school数据库分片功能
> db.runCommand({enablesharding:"school"}};
//指定school数据库需要分片的集合和片键
> db.runCommand({shardcollection:"school.user",key:{id:1}})
//查看状态
> db.user.stats()
另外有需要云服务器可以了解下创新互联cdcxhl.cn,海内外云服务器15元起步,三天无理由+7*72小时售后在线,公司持有idc许可证,提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案,具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势,专为企业上云打造定制,能够满足用户丰富、多元化的应用场景需求。