运行hadoop2 自带的圆周率计算方法时,报错,找了半天,原来是在配置hadoop临时目录时,没有给权限,找到配置的hadoop临时目录文件夹,修改权限即可

Application application_1548242073562_0005 failed 2 times due to AM Container for appattempt_1548242073562_0005_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2019-01-23 20:50:53.319]Exception from container-launch.
Container id: container_1548242073562_0005_02_000001
Exit code: 1
[2019-01-23 20:50:53.321]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.

计算少的时,比如1 1,不报错,如果计算稍微大的数字,如10 10 就直接报下面的异常

19/01/24 10:20:32 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
java.io.IOException: java.net.ConnectException: Your endpoint configuration is wrong; For more details see: http://wiki.apache.org/hadoop/UnsetHostnameOrPort
at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:344)
at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:429)
at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:804)
at org.apache.hadoop.mapreduce.Job$1.run(Job.java:331)
at org.apache.hadoop.mapreduce.Job$1.run(Job.java:328)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893)
at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:328)
at org.apache.hadoop.mapreduce.Job.isSuccessful(Job.java:624)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1603)
at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:307)
at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:360)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:368)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:244)
at org.apache.hadoop.util.RunJar.main(RunJar.java:158)
Caused by: java.net.ConnectException: Your endpoint configuration is wrong; For more details see: http://wiki.apache.org/hadoop/UnsetHostnameOrPort
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:824)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:750)
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1511)
at org.apache.hadoop.ipc.Client.call(Client.java:1453)
at org.apache.hadoop.ipc.Client.call(Client.java:1363)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy15.getJobReport(Unknown Source)
at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:325)
... 27 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:690)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:794)
at org.apache.hadoop.ipc.Client$Connection.access$3600(Client.java:412)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1568)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
... 36 more

  后来在这里找到了答案https://blog.csdn.net/zhouyan8603/article/details/47398245,,hadoop的datanode需要访问namenode的jobhistory server,如果没有修改,则默认为0.0.0.0:10020,则可以修改mapred-site.xml文件:

<property>
    <name>mapreduce.jobhistory.address</name>
       <!-- 配置实际的Master主机名和端口-->
    <value>0.0.0.0:10020</value>
</property> <property>
    <name>mapreduce.jobhistory.webapp.address</name>
       <!-- 配置实际的Master主机名和端口-->
    <value>0.0.0.0:19888</value>
</property>

然后启动historyserver,再尝试10 10  完成

$HADOOP_HOME/sbin/mr-jobhistory-daemon.sh start historyserver

运行hadoop自带的计算圆周率异常的更多相关文章

  1. Hadoop_05_运行 Hadoop 自带 MapReduce程序

    1. MapReduce使用 MapReduce是Hadoop中的分布式运算编程框架,只要按照其编程规范,只需要编写少量的业务逻辑代码即可实现 一个强大的海量数据并发处理程序 2. 运行Hadoop自 ...

  2. linux下在eclipse上运行hadoop自带例子wordcount

    启动eclipse:打开windows->open perspective->other->map/reduce 可以看到map/reduce开发视图.设置Hadoop locati ...

  3. Hadoop(1)---运行Hadoop自带的wordcount出错问题。

    在hadoop2.9.0版本中,对namenode.yarn做了ha,随后在某一台namenode节点上运行自带的wordcount程序出现偶发性的错误(有时成功,有时失败),错误信息如下: // : ...

  4. 如何运行Hadoop自带的例子

    bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.4.jar wordcount /WordCount/WordC ...

  5. 运行hadoop自带的wordcount例子程序

    1.准备文件 [root@master ~]# cat input.txt hello java hello python hello c hello java hello js hello html ...

  6. 在命令行中运行Hadoop自带的WordCount程序

    1.启动所有的线程服务 start-all.sh 记得要查看线程是否启动 jps 2.在根目录创建 wordcount.txt 文件 放置一些数据 3.创建  hdfs dfs -mkdir /文件夹 ...

  7. 运行Spark提供的计算圆周率的示例程序

    1.启动Spark服务 因为spark是依赖于hadoop提供的分布式文件系统的,所以在启动spark之前,先确保hadoop在正常运行. 在hadoop正常运行的情况下,在master(也就是had ...

  8. hadoop自带例子wordcount的具体运行步骤

    1.在hadoop所在目录“usr/local”下创建一个文件夹input root@ubuntu:/usr/local# mkdir input 2.在文件夹input中创建两个文本文件file1. ...

  9. 【爬坑】运行 Hadoop 的 MapReduce 示例卡住了

    1. 问题说明 在以伪分布式模式运行 Hadoop 自带的 MapReduce 示例,卡在了 Running job ,如图所示 2. 解决过程 查看日志没得到有用的信息 再次确认配置信息没有错误信息 ...

随机推荐

  1. AJPFX总结heap和stack有什么区别?

    栈是后进先出的线性表结构,存取速度比堆快.创建对象的时候new一个对象,引用存在栈上具体的内容存在堆上. 栈与堆都是Java用来在RAM中存放数据的地方.与C++不同,Java自动管理栈和堆,程序员不 ...

  2. AJPFX总结集合的概念

    //java 中集合的概述==========================================================        集合的概念:              为 ...

  3. AJPFX:实现递归统计文件夹的总大小

    class Statistical {    public static void main(String[] args) {        Scanner sc = new Scanner(Syst ...

  4. 阻止JEB 1.5频繁弹窗的办法

    偶尔才用一次的JEB, 出现 “Controller没有响应或者无法访问, JEB即将终止.” 也懒得去逆了.直接用ProcessHacker找到对应的线程挂起即可.当然这只是临时的办法..我也只是临 ...

  5. UVA 11020 Efficient Solutions (BST,Splay树)

    题意:给n个坐标.一个坐标(x,y)若有无存在的坐标满足x1<x && y1<=y  或  x1<=x && y1<y 时,此坐标(x,y)是就 ...

  6. LibreOJ #109. 并查集

    题目描述 这是一道模板题. 维护一个 nnn 点的无向图,支持: 加入一条连接 uuu 和 vvv 的无向边 查询 uuu 和 vvv 的连通性 由于本题数据较大,因此输出的时候采用特殊的输出方式:用 ...

  7. Docker 启动时报错:iptables:No chain/target/match by the name

    重新启动docker后,就好了. service docker restart

  8. Python3简明教程(十四)—— Collections模块

    collections 是 Python 内建的一个集合模块,提供了许多有用的集合类. 在这个实验我们会学习 Collections 模块.这个模块实现了一些很好的数据结构,它们能帮助你解决各种实际问 ...

  9. uva1608 Non-boring sequences

    某个序列找到唯一元素后,判断被分成的两边的序列即可问题在于找到唯一元素连续序列,重复元素的问题:感觉很有一般性 查找相同元素用map,last,next存上一个相同元素的位置复杂度计算有点思考:记录l ...

  10. BZOJ2120 数颜色(树套树)

    B. 数颜色 题目描述 墨墨购买了一套N支彩色画笔(其中有些颜色可能相同),摆成一排,你需要回答墨墨的提问.墨墨会像你发布如下指令:1. Q L R代表询问你从第L支画笔到第R支画笔中共有几种不同颜色 ...