二十九、rsync+inotity实时监控同步工具

一、场景应用：                           

        客户通过url访问资源（查询，下载等），并发量是非常高的，所以运用负载均衡分担web服务器的压力，在后端连接不同的NFS备份服务器，同样也是分担压力；那么在同步nfs服务器上的资源的时候，我们需要实时的同步到备份服务器上，这样用户才能使用这些资源，传统的定时任务，我们知道最快1分钟，同步一次，这是无法忍受的，所以我们用inotity进行实时的同步。

二、inotify+rsync组合的起源

Rsync远程同步工具可以进行数据的同步，但是在数据量非常庞大的今天，如果要实现两边的数据一致，rsync是支持的，那么就要进行数据的对比，但是在对比中发现，变化的数据只是一小部分，而且对比又是比较耗时，所以这里就出现了rsync的瓶颈，inotify的出现，缓解rsync的不足之处，实现实时同步。

三、Inotify的工作机制

        Inotify是一种强大的，细粒度的，异步的文件系统事件监控机制。

 四、启发：inotify可以监控目录的变化，那么变化后，既然可以触发同步rsync，那么同样可以触发发送邮件、打电话等，用处多多！！！

五、安装inotify

   1.首先inotify的实现软件有很多，这里说两种：

1）inotify自身    简单                          2）sercync 国内软件开发，功能强大，可以做过滤

在性能上intofy大于sersyrc

2.安装前置条件：

1）版本必须大于2.6.13

2）有/proc/sys/fs/inotify

 

3.源码下载，源码安装：

• mkdir -p /home/oldboy/tools/
• cd  /home/oldboy/tools/
• wget http://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz --no-check-certificate
• tar xf inotify-tools-3.14.tar.gz
•      cd inotify-tools-3.14
• ./configure --prefix=/usr/local/inotify-tools-3.14（安装出错，因为没有安装yum install gcc）-----》配置参数的安装和安装目录的指定
•       [root@djw1 inotify-tools-3.14]# echo $?      0   ====》检查没有出错了！！！！
•       make&&make install  -->编译成机器认识的语言 make成功在进行make install 安装
• ls  -s   /usr/local/inotify-tools-3.14   /usr/local/inotify   ---->软链接

 六、配置参数 ：cd /usr/local/inotify

   

1)bin   inotify执行命令     2）include  inotify头文件    3）lib  动态链接的库文件（系统调用）    4）share 帮助文档

       有两个命令要注意；inotifywait（监听事件）、inotifywatch

       inotifywait配置参数：建议用的时候熟练，多多看帮忙文档，如下：

文档参数：

 [root@djw1 inotify]# ./bin/inotifywait --help

inotifywait 3.14
Wait for a particular event on a file or set of files.
Usage: inotifywait [ options ] file1 [ file2 ] [ file3 ] [ ... ]
Options:
        -h|--help       Show this help text.
        @<file>         Exclude the specified file from being watched.
        --exclude <pattern>
                        Exclude all events on files matching the
                        extended regular expression <pattern>.
        --excludei <pattern>
                        Like --exclude but case insensitive.
        -m|--monitor    Keep listening for events forever.  Without
                        this option, inotifywait will exit after one
                        event is received.
        -d|--daemon     Same as --monitor, except run in the background
                        logging events to a file specified by --outfile.
                        Implies --syslog.
        -r|--recursive  Watch directories recursively.
        --fromfile <file>
                        Read files to watch from <file> or `-' for stdin.
        -o|--outfile <file>
                        Print events to <file> rather than stdout.
        -s|--syslog     Send errors to syslog rather than stderr.
        -q|--quiet      Print less (only print events).
        -qq             Print nothing (not even events).
        --format <fmt>  Print using a specified printf-like format
                        string; read the man page for more details.
        --timefmt <fmt> strftime-compatible format string for use with
                        %T in --format string.
        -c|--csv        Print events in CSV format.
        -t|--timeout <seconds>
                        When listening for a single event, time out after
                        waiting for an event for <seconds> seconds.
                        If <seconds> is 0, inotifywait will never time out.
        -e|--event <event1> [ -e|--event <event2> ... ]
                Listen for specific event(s).  If omitted, all events are
                listened for.

Exit status:
        0  -  An event you asked to watch for was received.
        1  -  An event you did not ask to watch for was received
              (usually delete_self or unmount), or some error occurred.
        2  -  The --timeout option was given and no events occurred
              in the specified interval of time.

Events:
        access          file or directory contents were read
        modify          file or directory contents were written
        attrib          file or directory attributes changed
        close_write     file or directory closed, after being opened in
                        writeable mode
        close_nowrite   file or directory closed, after being opened in
                        read-only mode
        close           file or directory closed, regardless of read/write mode
        open            file or directory opened
        moved_to        file or directory moved to watched directory
        moved_from      file or directory moved from watched directory
        move            file or directory moved to or from watched directory
        create          file or directory created within watched directory
        delete          file or directory deleted within watched directory
        delete_self     file or directory was deleted
        unmount         file system containing file or directory unmounted

七、人工测试监听事件：inotifywait

     命令：   /usr/local/inotify/bin/inotifywait  -mrq --timefmt '%Y/%m/%d %H:%M' --format '%T %w%f' -e create,delete,close_write  /backup

 

这里要注意：close_write 监控的是写入文件的内容，只要写入就会被监听到，但是如果创造文件，也是一种写入，所以有create存在，就会有两个事件，如下：

八、简单脚本：

[root@djw1 scripts]# vim inotify.sh  
#!/bin/bash
path="/usr/local/inotify/bin/inotifywait"
$path -mrq --format '%w%f' -e create,delete,close_write  /backup |\
while read line
do
  rsync -az $line  rsync_backup@192.168.0.103::oldboy --password-file=/etc/rsync.password
done

测试成功！！！

九、inotity的缺点：

1）并发不能大于200个文件（10-100k的文件）