1、硬件配置

采用3台虚拟机

节点名称 IP地址 内存 硬盘 节点角色
node1 192.168.1.6 2GB 10GB NameNode、ResoucerManager、ZKFC、DataNode、NodeManager、JournalNode、QuorumPeerMain
node2 192.168.1.7 2GB 10GB NameNode、ResoucerManager、ZKFC、DataNode、NodeManager、JournalNode、QuorumPeerMain
node3 192.168.1.8 2GB 10GB DataNode、NodeManager、JournalNode、QuorumPeerMain

2、软件版本

软件 版本
JDK jdk-8u271
HADOOP hadoop-3.2.1
ZOOKEEPER zookeeper-3.5.8

3、准备工作

3.1、配置网络环境

参考 Hadoop完全分布式

3.2、安装JDK

参考 Hadoop完全分布式

3.3、安装ZOOKEEPER

参考 ZOOKEEPER完全分布式

[root@node1 opt]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.5.8/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED [root@node1 opt]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.5.8/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower [root@node2 opt]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.5.8/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED [root@node2 opt]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.5.8/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: leader [root@node3 opt]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.5.8/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED [root@node3 opt]# jps
7602 Jps
7562 QuorumPeerMain [root@node3 opt]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.5.8/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower

4、安装Hadoop

配置配置文件,具体见文末。

5、启动

(1)分别在 node1、node2、node3 节点上启动 JournalNode 进程:

[root@node1 hadoop-3.2.1]# bin/hdfs --daemon start journalnode
[root@node1 hadoop-3.2.1]# jps
7890 JournalNode
7629 QuorumPeerMain
7934 Jps [root@node2 hadoop-3.2.1]# bin/hdfs --daemon start journalnode
[root@node2 hadoop-3.2.1]# jps
7890 JournalNode
7629 QuorumPeerMain
7934 Jps [root@node3 hadoop-3.2.1]# bin/hdfs --daemon start journalnode
[root@node3 hadoop-3.2.1]# jps
7890 JournalNode
7629 QuorumPeerMain
7934 Jps

(2)在其中一台 NameNode 上运行格式化命令:

[root@node1 hadoop-3.2.1]# bin/hdfs namenode -format
....
2020-12-16 21:04:43,291 INFO common.Storage: Storage directory /opt/hadoop-3.2.1/dfs/namenode has been successfully formatted.
....

(3)没有格式化的 NameNode 通过在此机器上运行如下命令,将已格式化的 NameNode 上的元数据目录复制过来:

[root@node2 hadoop-3.2.1]# bin/hdfs namenode -bootstrapStandby
....
=====================================================
About to bootstrap Standby ID node2 from:
Nameservice ID: hadoopcluster
Other Namenode ID: node1
Other NN's HTTP address: http://node1:9870
Other NN's IPC address: node1/192.168.1.6:8020
Namespace ID: 1789437097
Block pool ID: BP-1119377207-192.168.1.6-1608123883253
Cluster ID: CID-72958c07-b2b2-46f8-a035-667159c67de9
Layout version: -65
isUpgradeFinalized: true
=====================================================
2020-12-16 21:11:41,726 INFO common.Storage: Storage directory /opt/hadoop-3.2.1/dfs/namenode has been successfully formatted.
2020-12-16 21:11:41,753 INFO common.Util: Assuming 'file' scheme for path /opt/hadoop-3.2.1/dfs/namenode in configuration.
2020-12-16 21:11:41,753 INFO common.Util: Assuming 'file' scheme for path /opt/hadoop-3.2.1/dfs/namenode in configuration.
2020-12-16 21:11:41,805 INFO namenode.FSEditLog: Edit logging is async:true
2020-12-16 21:11:41,908 INFO namenode.TransferFsImage: Opening connection to http://node1:9870/imagetransfer?getimage=1&txid=0&storageInfo=-65:1789437097:1608123883253:CID-72958c07-b2b2-46f8-a035-667159c67de9&bootstrapstandby=true
2020-12-16 21:11:42,009 INFO common.Util: Combined time for file download and fsync to all disks took 0.00s. The file download took 0.00s at 0.00 KB/s. Synchronous (fsync) write to disk of /opt/hadoop-3.2.1/dfs/namenode/current/fsimage.ckpt_0000000000000000000 took 0.00s.
2020-12-16 21:11:42,010 INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000000 size 399 bytes.
2020-12-16 21:11:42,056 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at node2/192.168.1.7
************************************************************/

(4)在两个 NameNode 节点上启动 NameNode 进程:

[root@node1 hadoop-3.2.1]# bin/hdfs --daemon start namenode
[root@node1 hadoop-3.2.1]# jps
8705 Jps
7666 QuorumPeerMain
8437 JournalNode
8605 NameNode [root@node2 hadoop-3.2.1]# bin/hdfs --daemon start namenode
[root@node2 hadoop-3.2.1]# jps
8112 Jps
7890 JournalNode
7629 QuorumPeerMain
8045 NameNode

(5)可以在任意 NameNode 所在的主机上运行如下命令来格式化 ZKFC,并在将要运行 NameNode 的机器上启动 ZKFC:

[root@node1 hadoop-3.2.1]# bin/hdfs zkfc -formatZK
····
2020-12-16 21:18:06,960 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/hadoopcluster in ZK.
····
[root@node1 hadoop-3.2.1]# bin/hdfs --daemon start zkfc
[root@node1 hadoop-3.2.1]# jps
8848 DFSZKFailoverController
7666 QuorumPeerMain
8898 Jps
8437 JournalNode
8605 NameNode [root@node2 hadoop-3.2.1]# bin/hdfs --daemon start zkfc
[root@node2 hadoop-3.2.1]# jps
7890 JournalNode
8230 Jps
8199 DFSZKFailoverController
7629 QuorumPeerMain
8045 NameNode

(6)启动 datanode

[root@node1 hadoop-3.2.1]# bin/hdfs --daemon start datanode
[root@node1 hadoop-3.2.1]# jps
8848 DFSZKFailoverController
7666 QuorumPeerMain
9010 Jps
8437 JournalNode
8951 DataNode
8605 NameNode [root@node2 hadoop-3.2.1]# bin/hdfs --daemon start datanode
[root@node2 hadoop-3.2.1]# jps
7890 JournalNode
8199 DFSZKFailoverController
7629 QuorumPeerMain
8045 NameNode
8398 Jps
8335 DataNode [root@node3 hadoop-3.2.1]# bin/hdfs --daemon start datanode
[root@node3 hadoop-3.2.1]# jps
7968 DataNode
8000 Jps
7841 JournalNode
7562 QuorumPeerMain

(7)启动 ResourceManager 与 NodeManager

[root@node1 hadoop-3.2.1]# sbin/start-yarn.sh start resoucermanager
Starting resourcemanagers on [ node1 node2]
上一次登录:三 12月 16 19:08:13 CST 2020从 desktop-0ahq4ftpts/2 上
Starting nodemanagers
上一次登录:三 12月 16 21:25:35 CST 2020pts/1 上
[root@node1 hadoop-3.2.1]# jps
8848 DFSZKFailoverController
9472 NodeManager
7666 QuorumPeerMain
8437 JournalNode
8951 DataNode
9627 Jps
9340 ResourceManager
8605 NameNode [root@node2 hadoop-3.2.1]# jps
7890 JournalNode
8199 DFSZKFailoverController
8552 NodeManager
8475 ResourceManager
7629 QuorumPeerMain
8045 NameNode
8861 Jps
8335 DataNode [root@node3 hadoop-3.2.1]# jps
7968 DataNode
7841 JournalNode
7562 QuorumPeerMain
8204 Jps
8093 NodeManager

(8) web 查看

6、问题

(1)配置完配置文件,执行 bin/hdfs namenode -format,出现 a shared edits dir must not be specified if HA is not enabled. 问题。

原因:配置文件中的这种属性 dfs.ha.namenodes.[nameservice ID] 的 [nameservice ID] 没有改成自己的名字。

注意:要配置的属性较多,仔细检查。

(2)配置完配置文件,执行 bin/hdfs namenode -format,出现 Unable to check if JNs are ready for formatting 问题。

要在各个结点执行 bin/hdfs --daemon start journalnode 先启动 journalnode。

7、配置文件

core-site.xml

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoopcluster</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>131072</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoop/tmp</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>node1:2181,node2:2181,node3:2181</value>
</property>
</configuration>

hdfs-site.xml

<configuration>
<property>
<name>dfs.nameservices</name>
<value>hadoopcluster</value>
</property>
<property>
<name>dfs.ha.namenodes.hadoopcluster</name>
<value>node1,node2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.hadoopcluster.node1</name>
<value>node1:8020</value>
</property>
<property>
<name>dfs.namenode.rpc-address.hadoopcluster.node2</name>
<value>node2:8020</value>
</property>
<property>
<name>dfs.namenode.http-address.hadoopcluster.node1</name>
<value>node1:9870</value>
</property>
<property>
<name>dfs.namenode.http-address.hadoopcluster.node2</name>
<value>node2:9870</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://node1:8485;node2:8485;node3:8485/hadoopcluster</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.hadoopcluster</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence</value>
<value>shell(/bin/true)</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_dsa</value>
</property>
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/opt/hadoop-3.2.1/jndata</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/opt/hadoop-3.2.1/dfs/namenode</value>
</property>
<property>
<name>dfs.hosts</name>
<value>/opt/hadoop-3.2.1/etc/hadoop/workers</value>
</property>
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
</configuration>

mapred-site.xml

<configuration>
<property>
<!-- 指定mapreduce框架为yarn方式 -->
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<!-- Directory where history files are written by MapReduce jobs -->
<name>mapreduce.jobhistory.intermediate-done-dir</name>
<value>/mr-history/tmp</value>
</property>
<property>
<!-- Directory where history files are managed by the MR JobHistory Server -->
<name>mapreduce.jobhistory.done-dir</name>
<value>/mr-history/done</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>node1:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>node1:19888</value>
</property>
</configuration>

yarn-site.xml

<configuration>
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>yarncluster</value>
</property>
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>node1</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>node2</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address.rm1</name>
<value>node1:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address.rm2</name>
<value>node2:8088</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address.rm1</name>
<value>node1:8033</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address.rm2</name>
<value>node2:8033</value>
</property>
<property>
<name>yarn.resourcemanager.address.rm1</name>
<value>node1:8032</value>
</property>
<property>
<name>yarn.resourcemanager.address.rm2</name>
<value>node2:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address.rm1</name>
<value>node1:8030</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address.rm2</name>
<value>node2:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address.rm1</name>
<value>node1:8031</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address.rm2</name>
<value>node2:8031</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address.rm1</name>
<value>node1:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address.rm2</name>
<value>node2:8088</value>
</property>
<property>
<name>hadoop.zk.address</name>
<value>node1:2181,node2:2181,node3:2181</value>
</property>
<property>
<name>yarn.resourcemanager.nodes.include-path</name>
<value>/opt/hadoop-3.2.1/etc/hadoop/workers</value>
</property>
<property>
<name>yarn.nodemanager.local-dirs</name>
<value>/opt/hadoop-3.2.1/tmp</value>
</property>
<property>
<name>yarn.nodemanager.log-dirs</name>
<value>/opt/hadoop-3.2.1/logs</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.application.classpath</name>
<value>/opt/hadoop-3.2.1/etc/hadoop:/opt/hadoop-3.2.1/share/hadoop/common/lib/*:/opt/hadoop-3.2.1/share/hadoop/common/*:/opt/hadoop-3.2.1/share/hadoop/hdfs:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/*:/opt/hadoop-3.2.1/share/hadoop/hdfs/*:/opt/hadoop-3.2.1/share/hadoop/mapreduce/lib/*:/opt/hadoop-3.2.1/share/hadoop/mapreduce/*:/opt/hadoop-3.2.1/share/hadoop/yarn:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/*:/opt/hadoop-3.2.1/share/hadoop/yarn/*</value>
</property>
</configuration>

worker

node1
node2
node3

hadoop环境搭建:高可用的更多相关文章

  1. Nginx+Keepalived(双机热备)搭建高可用负载均衡环境(HA)

    原文:https://my.oschina.net/xshuai/blog/917097 摘要: Nginx+Keepalived搭建高可用负载均衡环境(HA) http://blog.csdn.ne ...

  2. Nginx+Keepalived(双机热备)搭建高可用负载均衡环境(HA)-转帖篇

    原文:https://my.oschina.net/xshuai/blog/917097 摘要: Nginx+Keepalived搭建高可用负载均衡环境(HA) http://blog.csdn.ne ...

  3. 转 史上最详细的Hadoop环境搭建

    GitChat 作者:鸣宇淳 原文:史上最详细的Hadoop环境搭建 关注公众号:GitChat 技术杂谈,一本正经的讲技术 [不要错过文末活动哦] 前言 Hadoop在大数据技术体系中的地位至关重要 ...

  4. 搭建高可用mongodb集群(四)—— 分片(经典)

    转自:http://www.lanceyan.com/tech/arch/mongodb_shard1.html 按照上一节中<搭建高可用mongodb集群(三)-- 深入副本集>搭建后还 ...

  5. [转]搭建高可用mongodb集群(四)—— 分片

    按照上一节中<搭建高可用mongodb集群(三)—— 深入副本集>搭建后还有两个问题没有解决: 从节点每个上面的数据都是对数据库全量拷贝,从节点压力会不会过大? 数据压力大到机器支撑不了的 ...

  6. [转]搭建高可用mongodb集群(二)—— 副本集

    在上一篇文章<搭建高可用MongoDB集群(一)——配置MongoDB> 提到了几个问题还没有解决. 主节点挂了能否自动切换连接?目前需要手工切换. 主节点的读写压力过大如何解决? 从节点 ...

  7. 搭建高可用mongodb集群(四)—— 分片

    按照上一节中<搭建高可用mongodb集群(三)—— 深入副本集>搭建后还有两个问题没有解决: 从节点每个上面的数据都是对数据库全量拷贝,从节点压力会不会过大? 数据压力大到机器支撑不了的 ...

  8. 搭建高可用mongodb集群(三)—— 深入副本集内部机制

    在上一篇文章<搭建高可用mongodb集群(二)—— 副本集> 介绍了副本集的配置,这篇文章深入研究一下副本集的内部机制.还是带着副本集的问题来看吧! 副本集故障转移,主节点是如何选举的? ...

  9. 搭建高可用mongodb集群(二)—— 副本集

    在上一篇文章<搭建高可用MongoDB集群(一)——配置MongoDB> 提到了几个问题还没有解决. 主节点挂了能否自动切换连接?目前需要手工切换. 主节点的读写压力过大如何解决? 从节点 ...

  10. 通过LVS+Keepalived搭建高可用的负载均衡集群系统

    1. 安装LVS软件      (1)安装前准备操作系统:统一采用Centos6.5版本,地址规划如下: 服务器名 IP地址 网关 虚拟设备名 虚拟ip Director Server 192.168 ...

随机推荐

  1. 提升手持设备点击速度之touch事件带来的坑!

    前言 上周六,我将我们项目的click换成了tap事件,于是此事如梦魇一般折磨了我一星期!!! 经过我前仆后继的努力,不计代价的牺牲,不断的埋坑填坑,再埋坑的动作,最后悲伤的发现touch事件确实是个 ...

  2. python中%和format

    两者都是格式化字符串用的,前者是比较老的版本,现在已经不推荐,后者更强大一些 % In [22]: print '%s' % 'hello world' hello world In [23]: pr ...

  3. JavaScript、jQuery、AJAX、JSON 解释

  4. Java相对路径读取文件

    不管你是新手还是老鸟,在程序中读取资源文件总会遇到一些找不到文件的问题,这与Java底层的实现有关,不能算bug,只要方法得当,问题还是可以解决的. 项目的文件夹结构: repathtest ├─sr ...

  5. 如何修改android工程的包名?

    在我们android项目开发到一定的程度时由于需要,我们必须修改一下工程的包名,以便更好的发布我们的项目.但是在这个过程中有时候修改好了之后会出现一些错误.下面由小编一步步教你如何更改包名,和解决出现 ...

  6. Effective Java 第三版——30. 优先使用泛型方法

    Tips <Effective Java, Third Edition>一书英文版已经出版,这本书的第二版想必很多人都读过,号称Java四大名著之一,不过第二版2009年出版,到现在已经将 ...

  7. ValueObject

    ValueObject When programming, I often find it's useful to represent things as a compound混合物. A 2D co ...

  8. chattr的使用

    让某个文件只能往里面追加内容,不能删除,一些日志文件适用于这种操作: chattr +a /home/caolei/.bash_history 查看lsattr /home/caolei/.bash_ ...

  9. Using Spring.net in console application

    Download Spring.net in http://www.springframework.net/ Install Spring.NET.exe Create a console appli ...

  10. 荣耀 6 安装 SD 卡,提示:SD卡已安全移除

    先前买了个 荣耀6(购买链接),自带存储只有 16G,用来一段时间后,老是提示存储不足.后来发现是 微信 等软件占用了好多存储(缓存),, 好吧,在京东上买了个 64G 扩展卡(购买链接),安装过程如 ...