rsync+inotify百万级文件实时同步

实验环境：Centos7.4

目的：将源服务器的文件实时同步至目标服务器

源服务器：10.11.1.107

目标服务器：10.11.1.106

分别在两个节点安装rsync

yum -y install rsync

源服务器107安装inotify实时监控文件变化

wget http://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
tar zxf inotify-tools-3.14.tar.gz
cd inotify-tools-3.14
./configure --prefix=/usr/local/inotify
make -j2
make install
ln -s /usr/local/inotify/bin/inotifywait /usr/local/bin/

源服务器107密码认证文件

 cat /etc/rsync.pwd
123456

修改文件权限为600

chmod 600 /etc/rsync.pwd

目标服务器106密码认证文件,webrsync用户需要提前创建并设置密码

cat /etc/rsync.pwd
webrsync:123456

修改文件权限为600

chmod 600 /etc/rsync.pwd

目标服务器106配置文件

 cat /etc/rsyncd.conf
# /etc/rsyncd: configuration file for rsync daemon mode

# See rsyncd.conf man page for more options.

# configuration example:

# uid = nobody
# gid = nobody
# use chroot = yes
# max connections = 4
# pid file = /var/run/rsyncd.pid
# exclude = lost+found/
# transfer logging = yes
# timeout = 900
# ignore nonreadable = yes
# dont compress   = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2

# [ftp]
#        path = /home/ftp
#        comment = ftp export area

uid = 0
gid = 0
use chroot = no
max connections = 0            #0表示没有限制
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsyncd.lock
log file = /var/log/rsyncd.log
log format = %t %a %m %f %b
# exclude = lost+found/
# transfer logging = yes
timeout = 600                                           #时间设置600最好
reverse lookup = no
# ignore nonreadable = yes
# dont compress   = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2

[ayd01]                                                     #要同步的模块
path = /data/
read only = false
ignore errors                                            #忽略一些io错误
hosts allow = 10.11.1.0/24
auth users = webrsync
secrets file = /etc/rsync.pwd

源服务器107创建 rsync.sh脚本,以下所有操作都在107上执行

src=/data/
des=ayd01
rsync_passwd_file=/etc/rsync.pwd
ip1=10.11.1.106
user=webrsync
cd ${src}
/usr/local/bin/inotifywait -mrq --format  '%Xe %w%f' -e modify,create,delete,attrib,close_write,move ./ | while read file
do
        INO_EVENT=$(echo $file | awk '{print $1}')
        INO_FILE=$(echo $file | awk '{print $2}')
        echo "-------------------------------$(date)------------------------------------"
        echo $file
        if [[ $INO_EVENT =~ 'CREATE' ]] || [[ $INO_EVENT =~ 'MODIFY' ]] || [[ $INO_EVENT =~ 'CLOSE_WRITE' ]] || [[ $INO_EVENT =~ 'MOVED_TO' ]]         # 判断事件类型
        then
                echo 'CREATE or MODIFY or CLOSE_WRITE or MOVED_TO'
                rsync -avzcR --password-file=${rsync_passwd_file} $(dirname ${INO_FILE}) ${user}@${ip1}::${des}
        fi
        if [[ $INO_EVENT =~ 'DELETE' ]] || [[ $INO_EVENT =~ 'MOVED_FROM' ]]
        then
                echo 'DELETE or MOVED_FROM'
                rsync -avzR --delete --password-file=${rsync_passwd_file} $(dirname ${INO_FILE}) ${user}@${ip1}::${des}
        fi
        if [[ $INO_EVENT =~ 'ATTRIB' ]]
        then
                echo 'ATTRIB'
                if [ ! -d "$INO_FILE" ]
                then
                        rsync -avzcR --password-file=${rsync_passwd_file} $(dirname ${INO_FILE}) ${user}@${ip1}::${des}
                fi
        fi
done

排除指定目录同步脚本，首先创建需要排除inotify监控的文件列表

vim /etc/rsyslog.d/inotify_exclude.lst
download/
tmp/

最终shell脚本如下

#!/bin/bash
src=/data/
INOTIFY_EXCLUDE='(download|tmp|(.*/*\.log)$)'
RSYNC_EXCLUDE='/etc/rsyslog.d/inotify_exclude.lst'
des=ayd01
rsync_passwd_file=/etc/rsync.pwd
ip1=10.11.1.106
user=webrsync
cd ${src}
/usr/local/bin/inotifywait -mrq --format  '%Xe %w%f' --exclude ${INOTIFY_EXCLUDE} -e modify,create,delete,attrib,close_write,move ./ | while read file
do
        INO_EVENT=$(echo $file | awk '{print $1}')
        INO_FILE=$(echo $file | awk '{print $2}')
        echo "-------------------------------$(date)------------------------------------"
        echo $file
        if [[ $INO_EVENT =~ 'CREATE' ]] || [[ $INO_EVENT =~ 'MODIFY' ]] || [[ $INO_EVENT =~ 'CLOSE_WRITE' ]] || [[ $INO_EVENT =~ 'MOVED_TO' ]]         # 判断事件类型
        then
                echo 'CREATE or MODIFY or CLOSE_WRITE or MOVED_TO'
                rsync -avzcR --exclude-from=${RSYNC_EXCLUDE} --password-file=${rsync_passwd_file} $(dirname ${INO_FILE}) ${user}@${ip1}::${des}
        fi
        if [[ $INO_EVENT =~ 'DELETE' ]] || [[ $INO_EVENT =~ 'MOVED_FROM' ]]
        then
                echo 'DELETE or MOVED_FROM'
                rsync -avzR --exclude-from=${RSYNC_EXCLUDE} --delete --password-file=${rsync_passwd_file} $(dirname ${INO_FILE}) ${user}@${ip1}::${des}
        fi
        if [[ $INO_EVENT =~ 'ATTRIB' ]]
        then
                echo 'ATTRIB'
                if [ ! -d "$INO_FILE" ]
                then
                        rsync -avzcR --exclude-from=${RSYNC_EXCLUDE} --password-file=${rsync_passwd_file} $(dirname ${INO_FILE}) ${user}@${ip1}::${des}
                fi
        fi
done

后台运行sh脚本

 nohup sh /root/rsync.sh &

将脚本加入到开机启动

echo "nohup sh /root/rsync.sh &" >> /etc/rc.local

可以先执行一遍手动完全同步验证是否,建议添加到定时任务,每天凌晨1点执行一遍全量同步。保证数据完整性

0 1 * * * rsync -vzrtopg --delete --progress /data/ --exclude='tmp/' --exclude='download/'  webrsync@10.11.1.106::ayd01 --password-file=/etc/rsync.pwd

　　

注意同步的目录权限为755或属主为webrsync

参考：https://blog.csdn.net/yanzhenjingfan/article/details/88667263