MongoDB笔记: 常见问题

安装

在Centos7下以tgz形式安装.

下载安装包

现在最新的版本是4.2.2, 但是客户端Robot3T现在对4.2的支持不太好, 所以还是使用4.0版本, 下载的是最新的4.0.14: https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-4.0.14.tgz

安装步骤

# 解压
tar zxvf mongodb-linux-x86_64-rhel70-4.0.14.tgz
# 移动到/opt/mongodb
mv mongodb-linux-x86_64-rhel70-4.0.14 /opt/mongodb/
cd /opt/mongodb/
# 将其软链到 /opt/mongodb/latest
ln -s mongodb-linux-x86_64-rhel70-4.0.14 latest

在/opt/mongodb/latest/下创建配置文件 mongod.conf, 这里没有启用用户名验证, verbosity一开始可以设为2, 安装启动没问题再改回0

storage:
  dbPath: /data/mongodb/data
  journal:
    enabled: true
systemLog:
  destination: file
  path: "/data/mongodb/mongod.log"
  verbosity: 0
  logAppend: true
net:
  bindIp: "172.17.5.10"
  port: 27017

创建相应的用户, 目录, 并设置好权限

# 将mongodb的可执行文件软链到bash路径中
ln -s /opt/mongodb/latest/bin/* /usr/local/bin/
# 创建用户
useradd mongod
# 创建对应的目录并设置用户
cd /data/
mkdir -p mongodb/data
chown -R mongod:mongod mongodb/

创建systemd服务文件, 并添加到开机启动中

vi /lib/systemd/system/mongodb.service
systemctl list-unit-files
systemctl enable mongodb.service
systemctl start mongodb
systemctl status mongodb

mongodb.service的内容

[Unit]
Description=An object/document-oriented database
Documentation=man:mongod(1)
After=network.target

[Service]
User=mongod
Group=mongod
ExecStart=/usr/local/bin/mongod --quiet --config /opt/mongodb/latest/mongod.conf

[Install]
WantedBy=multi-user.target

备份脚本

备份后压缩, 并只保留最新的60个备份文件

#!/bin/sh

MAXIMUM_BACKUP_FILES=60
BASE_FOLDER="/data/backup/mongo_dump"
# mongodb config
DB_HOSTNAME="172.17.5.10"
DB_PORT="27017"
DB_USERNAME=""
DATABASES=(
	"rikyplus"
)
EXCLUDED_TABLES=(
)

#=========
echo "MongoDB Backup Task"

CURRENT_DATE=$(date +%Y%m%d-%H%M)
DUMP_FOLDER="${DB_HOSTNAME}_${CURRENT_DATE}"
BACKUP_FOLDER="${BASE_FOLDER}/${DUMP_FOLDER}"
mkdir $BACKUP_FOLDER

count=0
while [ "x${DATABASES[count]}" != "x" ];do
    count=$(( count + 1 ))
done
echo "[+] ${count} databases will be dumped..."

# do the dump
for DATABASE in ${DATABASES[@]};do
    echo "[+] Dumping: ${DATABASE}"

    IGNORED_TABLES_STRING=''
    for TABLE in "${EXCLUDED_TABLES[@]}";do
      IGNORED_TABLES_STRING+=" --excludeCollection=${TABLE}"
    done
    echo -n "    Began:  ";echo $(date)
    if $(mongodump -h ${DB_HOSTNAME}:${DB_PORT} -d ${DATABASE} -o ${BACKUP_FOLDER} ${IGNORED_TABLES_STRING});then
        echo "    Dumped successfully"
    else
        echo "    Failed dumping this database"
    fi
    echo -n "    Finished: ";echo $(date)
done

# Compress to tar ball
echo
echo "[+] Packaging and compressing the backup folder..."
if $(tar -czf ${BACKUP_FOLDER}.tar.gz -C ${BASE_FOLDER} ${DUMP_FOLDER});then
    rm -rf ${BACKUP_FOLDER}
    echo "    Compressed successfully"
  else
    echo "    Failed compressing this dump"
fi
echo -n "    Finished: ";echo $(date)

# Remove the oldest one
BACKUP_FILES_MADE=$(ls -l ${BASE_FOLDER}/*.tar.gz | wc -l)
BACKUP_FILES_MADE=$(( $BACKUP_FILES_MADE - 0 ))
echo
echo "[+] There are ${BACKUP_FILES_MADE} backup files"

if [ $BACKUP_FILES_MADE -gt $MAXIMUM_BACKUP_FILES ];then
    REMOVE_FILES=$(( $BACKUP_FILES_MADE - $MAXIMUM_BACKUP_FILES ))
	echo "[+] Remove ${REMOVE_FILES} old backup files."
	ALL_BACKUP_FILES=($(ls -t ${BASE_FOLDER}/*.tar.gz))
	EXPIRED_FILES=("${ALL_BACKUP_FILES[@]:${MAXIMUM_BACKUP_FILES}:${BACKUP_FILES_MADE}}")
	echo "[+] Removing the expired backup files..."
	for FILE in ${EXPIRED_FILES[@]};do
		rm -f ${FILE}
	done
fi

echo -n "    Finished: ";echo $(date)

　　

系统配置

设置ulimit

MongoDB的文件机制

• 每个Collection会单独创建一个数据文件(collection-xxxxxx.wt)
• 每个索引会单独创建一个文件(index-xxxxxx.wt)
• 每个Collection至少需要一个数据文件和一个默认索引(_id索引)

这样每创建1个Collection会占用2个文件句柄, 还需要包含MongoDB自身需要使用的句柄(内部文件如wiredTiger.wt之类, 加上网络连接), 所以Collection的数量是受系统限制的. MongoDB推荐的方式是分片, 不是分库分表. 会造成的问题比如重启时间会很长, 这不是MongoDB推荐的使用方式.

在句柄消耗达到系统上限后, 会报directory-sync: open: Too many open files错误, 并导致mongod崩溃退出. 增加限制需要修改/etc/security/limits.conf (或者 /etc/security/limits.d/20-nproc.conf), 增加或修改为以下内容

*          soft    nofile
*          hard    nofile
*          soft    nproc
*          hard    nproc
root       soft    nproc     unlimited

需要重启, 用 ulimit -n 检查