INFO hdfs.DFSClient: Exception in createBlockOutputStream java.net解决办法
自己安装好Hadoop2.7.x之后,发现dfs中的/bin/hadoop fs -put命令不能够使用,报错如下:
[hadoop@master bin]$ ./hadoop fs -put ../logs/123 /wc/mytemp
16/07/27 01:29:26 INFO hdfs.DFSClient: Exception in createBlockOutputStream
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1537)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1313)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1266)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:449)
16/07/27 01:29:26 INFO hdfs.DFSClient: Abandoning BP-555863411-172.16.95.100-1469590594354:blk_1073741825_1001
16/07/27 01:29:26 INFO hdfs.DFSClient: Excluding datanode DatanodeInfoWithStorage[172.16.95.101:50010,DS-ee00e1f8-5143-4f06-9ef8-b0f862fce649,DISK]
16/07/27 01:29:26 INFO hdfs.DFSClient: Exception in createBlockOutputStream
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1537)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1313)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1266)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:449)
16/07/27 01:29:26 INFO hdfs.DFSClient: Abandoning BP-555863411-172.16.95.100-1469590594354:blk_1073741826_1002
16/07/27 01:29:26 INFO hdfs.DFSClient: Excluding datanode DatanodeInfoWithStorage[172.16.95.102:50010,DS-eea51eda-0a07-4583-9eee-acd7fc645859,DISK]
16/07/27 01:29:26 WARN hdfs.DFSClient: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /wc/mytemp/123._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1). There are 2 datanode(s) running and 2 node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1547)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3107)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3031)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:724)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
at org.apache.hadoop.ipc.Client.call(Client.java:1475)
at org.apache.hadoop.ipc.Client.call(Client.java:1412)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:418)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy10.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1459)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1255)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:449)
put: File /wc/mytemp/123._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1). There are 2 datanode(s) running and 2 node(s) are excluded in this operation.
[hadoop@master bin]$ service firewall
The service command supports only basic LSB actions (start, stop, restart, try-restart, reload, force-reload, status). For other actions, please try to use systemctl.
到网上找资料,好不统一找到几个解决的办法,感觉值的参考的是该文:hadoop常见错误解决办法,在其第14条,他给出的解决办法如下:
I have resolved the issue:
What i did:
1) '/etc/init.d/iptables stop' -->stopped firewall
2) SELINUX=disabled in '/etc/selinux/config' file.-->disabled selinux
I worked for me after these two changes
需要补充的地方:
(1)此处CentOS7中,防火墙的方式改变,关闭防火墙改成service firewalld stop
(2)不仅是master中的firewall要关闭,所有的从节点(slave)中的防火墙也要关闭。
(3)检查所有的主机,/etc/selinux/config下的SELINUX,设置SELINUX=disable。
(4)建议关闭防火墙之后,再使用systemctl disable firewalld.service #禁止firewall开机启动
然后在进行put的时候,就成功了。截图如下:
另外一个人博主给出的解决方法如下:hadoop报错:hdfs.DFSClient: Exception in createBlockOutputStream, 大家可以尝试一下,可以这样说,新手一般都会挂掉,因为格式化namenode的时候,slave中的文件会和namenode不一致,造成DataNode节点启动失败,解决办法就是删除slave和master中所有的data节点,然后重新执行namenode格式化操作。(不只是自己水平有限还是问题不一样,这种方法初步判定为坑人)
INFO hdfs.DFSClient: Exception in createBlockOutputStream java.net解决办法的更多相关文章
- hadoop报错:hdfs.DFSClient: Exception in createBlockOutputStream
hadoop跑任务搞的好好的,后来把hadoop-dir移了一个位置,结果报错: java.io.EOFException: Premature EOF: no length prefix avail ...
- hdfs.DataStreamer: Exception in createBlockOutputStream
在上传文件至 HDFS 提示如下信息 [root@h136 jdk1.8.0_202]# hadoop fs -put javafx-src.zip / 19/04/11 23:19:36 INFO ...
- HDFS执行getDatanodeReport时权限不足的解决办法
通过JAVA获取HDFS的getDatanodeReport方法时,报权限不足的错误信息. org.apache.hadoop.ipc.RemoteException(org.apache.hadoo ...
- Error: IO_ERROR : java.io.IOException: Error while connecting Oozie server. No of retries = 5. Exception = Connection refused (Connection refused)解决办法(图文详解)
不多说,直接上干货! 问题详情 解决办法 Copy/Paste oozie.services property tag set from oozie-default.xml to oozie-site ...
- Eclipse遇到Initializing Java Tooling解决办法
可参考博客:http://liaojuncai.iteye.com/blog/1954113 打开Eclipse的时候莫名其妙的出现这个错误,进度条一直显示这个提示Initializing java ...
- Error: could not find java.dll 解决办法
Error: could not find java.dll 问题: 安装配置Java环境变量后,在命令行中运行java -version进行测试时却出现下面的问题: Error: opening r ...
- MyEclipse Servers视窗出现“Could not create the view: An unexpected exception was thrown”错误的解决办法
打开myeclipse所在的wordspace文件夹,在下面子文件夹 .metadata\.plugins\org.eclipse.core.runtime\.settings删除 com.genui ...
- eclipse导入web项目变成java项目解决办法
右键工程,properties-> Project Facets-> 点convert to faceted..连接 -> 把Dynamic Web Moudle勾上
- idea自动重置language level和java compiler解决办法:修改setting
maven工程: 错误: -source 1.6 中不支持 diamond 运算符. 尝试按网的的方式修改后,自动恢复,也在pom文件指定版本,依然不行. 后来发现:pom这样配置了: <plu ...
随机推荐
- 苹果mac版微软官方远程连接工具下载Microsoft Remote Desktop For Mac
官网beta版本,不用再到处找包了. ** 点我访问https://rink.hockeyapp.net/apps/5e0c144289a51fca2d3bfa39ce7f2b06/**
- Java中static的使用
static 使用之静态变量: 大家都知道,我们可以基于一个类创建多个该类的对象,每个对象都拥有自己的成员,互相独立.然而在某些时候,我们更希望该类所有的对象共享同一个成员.此时就是 static 大 ...
- 【c语言】将正数变成相应的负数,将负数变成相应的正数
<pre name="code" class="cpp">// 将正数变成相应的负数,将负数变成相应的正数 #include <stdio.h ...
- curl使用例子
地址:http://phpbook.phpxy.com/34771 参考:http://php.net/manual/zh/function.curl-setopt.php 我们将curl的步骤分为以 ...
- 详略。。设计模式2——单例变形(多例).。。。studying
★ 缓存在单例中的使用("单例+缓存"技术) 缓存在编程中使用非常频繁,有着非常关键的数据,它可以帮助程序实现以空间换取时间, 通常被设计成整个应用程序所共享的一个空间,现要求实现 ...
- iOS项目开发实战——自己定义圆形进度提示控件
iOS中默认的进度条是水平方向的进度条,这往往不能满足我们的需求. 可是我们能够自己定义类似的圆形的进度提示控件,主要使用iOS中的画图机制来实现. 这里我们要实现一个通过button点击然后圆形进度 ...
- 关于C# wpf DataGrid单元格双击设置单元格内容
1.我是使用了 visual stadio 2015, 用的C# WPF写个工具,但是发现wpf原生没有涉及表格的东西(类似 winform·的DataGridView),所以使用的是toolkit工 ...
- 【ASK】git使用中出现Permission denied (publickey).
好久没有用git了,今天突然执行了一下 $git submodule update --init --recursive =============================== 结果出现如下提 ...
- 【ask】Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral.
1.周五对nodejs tcp长连接服务器框架进行压测,一切开始比较常规(没什么特殊问题). 2.突然手一哆嗦,把压测用的客户端群一起关闭了. 3.这个时候nodejs的服务器爆出了"Cau ...
- android webview css z-index属性无效
在做android的web页面嵌入的时候,当使用css的z-index设置重叠div失败: 查询google说设置 -webkit-transform:translateZ(0) canvas{ -w ...