yarn.resourcemanager.ha.id设置
resourcemanager启动报错,其中一个启动成功,另一个启动报8088端口被成功启动的rm占用
2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/hadoop/lib/native
2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.32-504.el6.x86_64
2016-11-18 17:08:49,479 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=yarn
2016-11-18 17:08:49,479 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/var/lib/hadoop-yarn
2016-11-18 17:08:49,479 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/usr/lib/hadoop-yarn
2016-11-18 17:08:49,480 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=node01:2181,node02:2181,node03:2181 sessionTimeout=10000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@729b9707
2016-11-18 17:08:49,501 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server node02/172.16.145.112:2181. Will not attempt to authenticate using SASL (unknown error)
2016-11-18 17:08:49,507 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to node02/172.16.145.112:2181, initiating session
2016-11-18 17:08:49,518 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server node02/172.16.145.112:2181, sessionid = 0x2587635b7620006, negotiated timeout = 10000
2016-11-18 17:08:49,521 INFO org.apache.hadoop.ha.ActiveStandbyElector: Session connected.
2016-11-18 17:08:49,544 INFO org.apache.hadoop.ha.ActiveStandbyElector: Successfully created /yarn-leader-election/myclusterrm in ZK.
2016-11-18 17:08:49,549 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to standby state
2016-11-18 17:08:49,549 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to standby state
2016-11-18 17:08:49,642 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2016-11-18 17:08:49,648 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.resourcemanager is not defined
2016-11-18 17:08:49,666 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2016-11-18 17:08:49,677 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster
2016-11-18 17:08:49,677 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static
2016-11-18 17:08:49,677 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs
2016-11-18 17:08:49,678 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster
2016-11-18 17:08:49,678 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2016-11-18 17:08:49,678 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2016-11-18 17:08:49,684 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /cluster/*
2016-11-18 17:08:49,684 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2016-11-18 17:08:49,701 INFO org.apache.hadoop.http.HttpServer2: HttpServer.start() threw a non Bind IOException
java.net.BindException: Port in use: 172.16.145.114:8088
at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:890)
at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:826)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:273)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:903)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1014)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1145)
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:444)
at sun.nio.ch.Net.bind(Net.java:436)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:885)
... 6 more
标红的IP:172.16.145.114为成功启动的resourcemanager,上面的报错信息是启动失败的rm(172.16.145.115)报的错
错误显示114的8088端口已经在使用,跟我115有毛关系啊。
检查配置文件发现 两个rm中的yarn-site.xml中的 yarn.resourcemanager.ha.id设置成了同一个rm的主机名
解决办法:rm1的yarn-site的yarn.resourcemanager.ha.id设置成rm1
rm2的yarn-site的yarn.resourcemanager.ha.id设置成rm2
<!--在node04上配rm1,node05上配rm2,slave节点不用配这个属性-->
<property>
<name>yarn.resourcemanager.ha.id</name>
<value>rm1</value>
</property>
错误总结:一直以为两个resourcemanager配置文件是一样的,没有注意到这个属性。
对于这个属性两个rm的yarn-site设置不一样,而且slave中不需要配,其他属性暂时还发现,所有节点保持一致就行了
yarn.resourcemanager.ha.id设置的更多相关文章
- 通过tarball形式安装HBASE Cluster(CDH5.0.2)——配置分布式集群中的YARN ResourceManager 的HA
<?xml version="1.0"?> <!-- Licensed under the Apache License, Version 2.0 (the &q ...
- Hadoop官方文档翻译—— YARN ResourceManager High Availability 2.7.3
ResourceManager High Availability (RM高可用) Introduction(简介) Architecture(架构) RM Failover(RM 故障切换) Rec ...
- 【Hadoop学习】Apache Hadoop ResourceManager HA
简介 本向导简述了YARN资源管理器的HA,并详述了如何配置并使用该特性.RM负责追踪集群中的资源,并调度应用程序(如MapReduce作业).Hadoop2.4以前,RM是YARN集群中的单点故障. ...
- Hadoop-2.3.0-cdh5.0.1完全分布式环境搭建(NameNode,ResourceManager HA)
编写不易,转载请注明(http://shihlei.iteye.com/blog/2084711)! 说明 本文搭建Hadoop CDH5.0.1 分布式系统,包括NameNode ,Resource ...
- Hadoop(HDFS,YARN)的HA集群安装
搭建Hadoop的HDFS HA及YARN HA集群,基于2.7.1版本安装. 安装规划 角色规划 IP/机器名 安装软件 运行进程 namenode1 zdh-240 hadoop NameNode ...
- Hadoop NameNode HA 和 ResourceManager HA
1.集群规划 1.1 规划说明 hadoop1 cluster1 nameNode hadoop2 cluster1 nameNodeStandby ZooKeeper ResourceManager ...
- 安装部署Apache Hadoop (完全分布式模式并且实现NameNode HA和ResourceManager HA)
本节内容: 环境规划 配置集群各节点hosts文件 安装JDK1.7 安装依赖包ssh和rsync 各节点时间同步 安装Zookeeper集群 添加Hadoop运行用户 配置主节点登录自己和其他节点不 ...
- HDFS 和 YARN 的 HA 故障切换【转】
来源:https://blog.csdn.net/u011414200/article/details/50336735 一 非 HDFS HA 集群转换成 HA 集群二 HDFS 的 HA 自动切换 ...
- Hadoop 2.6.0 Namenode HA,ResourceManager HA
先启动所有的zookeeper zkServer.sh start 在所有节点上启动JournalNode: sbin/hadoop-daemon.sh start journalnode 格式化第一 ...
随机推荐
- Leetcode: Longest Palindromic Substring. java
Given a string S, find the longest palindromic substring in S. You may assume that the maximum lengt ...
- ISO13485给企业带来的益处
1.ISO13485变强制性认证,日益受到欧美和中国政府机构的重视,有利于消除国际贸易中的技术壁垒,是进入国际市场的通行证: 2.提高和改善企业的管理水平,增加企业的知名度: 3.提高和保证产品的质量 ...
- OC基础之方法和参数的命名规范
以前学过C/C++/Java/C#语言的童鞋可能刚开始对于OC的方法和参数的命名规范大为不爽 举例来说,如下一个OC方法: - (void)tableView:(UITableView *)table ...
- Visual Studio原生开发的10个调试技巧(一)
最近碰巧读了Ivan Shcherbakov写的一篇文章,<11个强大的Visual Studio调试小技巧>.这篇文章只介绍了一些有关Visual Studio的基本调试技巧,但是还有其 ...
- [置顶] Android开发之MediaPlayerService服务详解(一)
前面一节我们分析了Binder通信相关的两个重要类:ProcessState 和 IPCThreadState.ProcessState负责打开Binder 驱动,每个进程只有一个.而 IPCThre ...
- C语言阶乘和求闰年
#include<stdio.h> void main(){ int i,a,s=1; scanf("%d",&a); for(i=1;i<=a;i++) ...
- Java中类的初始化顺序
一.一个类的初始化顺序(没继承情况) 规则: 1.静态变量>普通变量>构造方法 2.变量定义的顺序决定初始化的顺序 3.静态变量和静态块是一样的,普通变量和非静态块是一样的,即能够把 ...
- Javascript模块化编程系列三: CommonJS & AMD 模块化规范描述
CommonJS Module 规范 CommonJS 的模块化规范描述在Modules/1.1.1 中 目前实现此规格的包有: Yabble,CouchDB,Narwhal (0.2), Wakan ...
- DP之矩阵连乘问题
最优二叉查找树的一道思考习题 同最优二叉查找树一样,矩阵连乘问题也是一个卡特兰数问题(其动态规划的构造过程都很像) 分析解答: a,铺垫的数学知识首先要搞清楚矩阵相乘是怎么乘的: 1)对于连续的n个矩 ...
- pthread_mutex_t
在Linux中使用线程 http://blog.csdn.net/jiajun2001/article/details/12624923 :LINUX就是这个范围作者 原创作品,允许转载,转载时 ...