[hadoop][基本原理]zookeeper场景使用

代码：https://github.com/xufeng79x/ZkClientTest

1. 简介

　　zookeeper的特性决定他适用到某些场景非常合适，比如典型的应用场景：

　　1.集群管理（Group Membership）

　　2.统一命名服务（Name Service）

　　3.配置管理（Configuration Management）

　　4.共享锁（Locks）

　　5.队列管理

2.集群管理

　　　在hadoop中主备节点的概念大家都应该不默认，比如HBase中，我们可以启动多个master节点，但是在某时刻只有一个主master，当这个主master节点退出后

其他的背master将会去争取成为主master。一般流程为：

　　如上，一开始的时候多个master都会去zookeeper服务上创建相同的临时路径，其中某个master会节点会成功创建，那么这个节点就是主master了，其他创建不成功的将会观察这个临时路径。

当主master由于宕机或者网络原因失去和zookeeper的连接（session）那么这个临时节点就会被zookeeper删除，随后其他备master将会感知到，再去争抢这个临时路径的创建权。

　　如下代码利用多线程来模拟多个master的场景：

public class Master implements Runnable {
    // master节点路径
    private String masterPath = null;

    // 当前master的信息
    private MasterInfo myinfo = null;

    // 当前主master的信息
    private MasterInfo activeMasterInfo = null;

    // 控制当前master的运行时间
    private long runningTime = ;

    // 监控handler
    private IZkDataListener masterPathListener = null;

    // 当前session
    private ZkClient zc = new ZkClient("xufeng-1:2181,xufeng-2:2181,xufeng-3:2181", , , new SerializableSerializer());
    public Master(MasterInfo info, long runningTime, String masterPath)
    {
        this.myinfo = info;
        this.runningTime = runningTime;
        this.masterPath = masterPath;
        this.masterPathListener = new IZkDataListener() {
            // 当节点被删除的时候触发此方法
            public void handleDataDeleted(String dataPath) throws Exception {
                attackMaster();
            }

            public void handleDataChange(String dataPath, Object data) throws Exception {
                // do nothing

            }
        };

        // 订阅节点
        zc.subscribeDataChanges(masterPath, masterPathListener);
    }

    public void run() {
        attackMaster();
        try {
            Thread.sleep(runningTime);
        } catch (InterruptedException e) {
            // do nothing
        }
    }

    // 去争抢这个节点创建（注册）
    private void attackMaster()
    {
        try {
            // 注册节点
            zc.create(masterPath, myinfo, CreateMode.EPHEMERAL);
            // 如果当前注册成功了，那么他就是主master
            activeMasterInfo = myinfo;
            System.out.println("the active master is : " + activeMasterInfo);
        }
        catch (ZkNodeExistsException e)
        {
            // 当节点已经被其他master注册了
            activeMasterInfo = zc.readData(masterPath);
            // 当无法读取到节点信息则认为其他master可能宕机了，再去抢注
            if (null == activeMasterInfo)
            {
                attackMaster();
            }
            else{
                System.out.println(activeMasterInfo + " has become active! " + myinfo +  " wait for next time to be active!");
            }
        }
        catch (Exception e)
        {
            // 当发生其他错误的时候，不去例会
        }

    }
}

　　当我们建立多个线程去启动的时候，多个master就会去抢注/master节点：

the active master is : MasterInfo [id=C, name=masterC]
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=G, name=masterG] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=F, name=masterF] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=B, name=masterB] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=E, name=masterE] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=A, name=masterA] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=H, name=masterH] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=D, name=masterD] wait for next time to be active!

　　这个时候我们模拟网络问题，手动地去删除/master节点多次的时候，各个master感知到节点删除会再次抢注：

the active master is : MasterInfo [id=C, name=masterC]
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=G, name=masterG] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=F, name=masterF] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=B, name=masterB] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=E, name=masterE] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=A, name=masterA] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=H, name=masterH] wait for next time to be active!
MasterInfo [id=C, name=masterC] has become active! MasterInfo [id=D, name=masterD] wait for next time to be active!
the active master is : MasterInfo [id=B, name=masterB]
MasterInfo [id=B, name=masterB] has become active! MasterInfo [id=F, name=masterF] wait for next time to be active!
MasterInfo [id=B, name=masterB] has become active! MasterInfo [id=H, name=masterH] wait for next time to be active!
MasterInfo [id=B, name=masterB] has become active! MasterInfo [id=G, name=masterG] wait for next time to be active!
MasterInfo [id=B, name=masterB] has become active! MasterInfo [id=E, name=masterE] wait for next time to be active!
MasterInfo [id=B, name=masterB] has become active! MasterInfo [id=C, name=masterC] wait for next time to be active!
MasterInfo [id=B, name=masterB] has become active! MasterInfo [id=A, name=masterA] wait for next time to be active!
MasterInfo [id=B, name=masterB] has become active! MasterInfo [id=D, name=masterD] wait for next time to be active!
the active master is : MasterInfo [id=E, name=masterE]
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=C, name=masterC] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=H, name=masterH] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=F, name=masterF] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=A, name=masterA] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=D, name=masterD] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=B, name=masterB] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=G, name=masterG] wait for next time to be active!

　　我们可以看到每一次抢注成功的master都不一样，这样如果是由于网络问题而不是当前主master真的宕机了，那么会造成不必要的主备切换。所以说我们还可以进行如下的程序优化：　　

　　如上图，当由于网络原因原来的主master其实并没有宕机，那么为了减小由于主备切换带来的集群抖动，可以让其他备master延迟一定时间去争抢，而当前的主master则马上去争抢。所以说即使

成为了主master也要观察这个临时路径。

　　所以在程序中我们可以在抢注的时候判断如果当前master并不是之前的主master，则延迟一定时间去抢注，使得当前主master能够再次成功的抢的节点，具体优化代码如下：

            // 当节点被删除的时候触发此方法
            public void handleDataDeleted(String dataPath) throws Exception {
                if (null != activeMasterInfo && activeMasterInfo.equals(myinfo))
                {
                    // 如果当前master就是主master的时候，直接去抢注
                    attackMaster();
                }
                else
                {
                    // 如果不是则延迟5秒去抢注，给原先的主master一个机会
                    delayExector.schedule(new Runnable() {

                        public void run() {
                            attackMaster();

                        }
                    }, , TimeUnit.SECONDS);
                }

            }

　　手动的去删除/master节点，结果：masterE始终很稳定的抢注到了/master节点。

the active master is : MasterInfo [id=E, name=masterE]
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=H, name=masterH] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=C, name=masterC] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=A, name=masterA] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=B, name=masterB] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=D, name=masterD] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=F, name=masterF] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=G, name=masterG] wait for next time to be active!
the active master is : MasterInfo [id=E, name=masterE]
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=B, name=masterB] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=G, name=masterG] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=F, name=masterF] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=D, name=masterD] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=H, name=masterH] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=C, name=masterC] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=A, name=masterA] wait for next time to be active!
the active master is : MasterInfo [id=E, name=masterE]
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=F, name=masterF] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=H, name=masterH] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=C, name=masterC] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=A, name=masterA] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=D, name=masterD] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=B, name=masterB] wait for next time to be active!
MasterInfo [id=E, name=masterE] has become active! MasterInfo [id=G, name=masterG] wait for next time to be active!

小结：

　　除了以上抢注某个临时节点的方式去进行主备切换实现外，我们也可以让每一个master在某个永久节点下各自注册自己的临时节点（CreateMode.EPHEMERAL_SEQUENTIAL）

方式，当观察到这个永久节点下znode有变动的时候，查看自己是不是后缀最小的一个，是，则将变成主master。

　　除了主备切换场景外，集群管理中的节点发现，任务分发，也同样可以有zookeeper来处理，这种灵活的使用方式可以解决很多分布式场景的问题。

3.其他

// 略