hadoop的单机部署很简单也不容易出错，但是对生产环境的价值和意义不大，但是可以快速用于开发。

部署hadoop的错误原因不少，并且很奇怪。

比如，用户名不同，造成客户端和服务器通讯产生认证失败的错误，客户端，服务器各节点的用户名应当是一致的，并且个节点应该建立ssh的无认证登陆。

一、出现下面错误：

13/07/09 13:57:07 INFO ipc.Client: Retrying connect to server: master/192.168.2.200:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

或

java.net.ConnectException: Call to master/192.168.2.200:9000 failed on connection exception: java.net.ConnectException: Connection refused

        at org.apache.hadoop.ipc.Client.wrapException(Client.java:1136)

        at org.apache.hadoop.ipc.Client.call(Client.java:1112)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.renewLease(Unknown Source)

        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.renewLease(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:379)

        at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:378)

        at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:400)

        at org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69)

        at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273)

        at java.lang.Thread.run(Thread.java:722)

Caused by: java.net.ConnectException: Connection refused

        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:719)

        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)

        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511)

        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481)

        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:453)

        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:579)

        at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:202)

        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1243)

        at org.apache.hadoop.ipc.Client.call(Client.java:1087)

        ... 14 more

是客户端无法连接服务器造成的，可能是服务器没有启动或者启动了防火墙。

二、出现下面错误：

13/07/09 13:57:36 ERROR hdfs.DFSClient: Failed to close file /tmp/web304069331.log

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web304069331.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

原因很多，namenode节点和datanode节点不能正常通讯造成的。根据查询启动日志看到datanode节点没有办法解析机器名造成的，所以修改/etc/hostname和/etc/hosts文件。

三、出现一下错误：

13/07/09 13:59:01 INFO hdfs.DFSClient: Exception in createBlockOutputStream 192.168.2.201:50010 java.net.SocketTimeoutException: 63000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=/192.168.2.201:50010]

13/07/09 13:59:01 INFO hdfs.DFSClient: Abandoning blk_-6965665250189110825_1679

13/07/09 13:59:01 INFO hdfs.DFSClient: Excluding datanode 192.168.2.201:50010

13/07/09 13:59:01 WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web465901718.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

13/07/09 13:59:01 WARN hdfs.DFSClient: Error Recovery for blk_-6965665250189110825_1679 bad datanode[0] nodes == null

13/07/09 13:59:01 WARN hdfs.DFSClient: Could not get block locations. Source file "/tmp/web465901718.log" - Aborting...

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web465901718.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

^C七月 09, 2013 2:00:46 下午 org.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler ["http-bio-8080"]

七月 09, 2013 2:00:46 下午 org.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler ["ajp-bio-8009"]

七月 09, 2013 2:00:46 下午 org.apache.catalina.core.StandardService stopInternal

INFO: Stopping service Catalina

13/07/09 14:00:46 ERROR hdfs.DFSClient: Failed to close file /tmp/web465901718.log

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web465901718.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

七月 09, 2013 2:00:46 下午 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads

是因为客户端与数据节点通讯失败造成的。客户端程序应该能够和所有的节点通讯才能保证数据的传输正常。

四、hadoop namenode -format时出现：

Format aborted in /home/cnsworder/hadoop/name

删除掉这个文件夹重新格式化。

hadoop部署错误的更多相关文章

  1. Hadoop部署方式-本地模式(Local (Standalone) Mode)

    Hadoop部署方式-本地模式(Local (Standalone) Mode) 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. Hadoop总共有三种运行方式.本地模式(Local ...

  2. hadoop常见错误总结三

    问题导读:1.... could only be replicated to 0 nodes, instead of 1 ...可能的原因是什么?2.Error: java.lang.NullPoin ...

  3. hadoop部署小结的命令

    hadoop部署总结的命令 学习笔记,转自:hadoop部署总结的命令http://www.aboutyun.com/thread-5385-1-1.html(出处: about云开发)

  4. Tomcat学习笔记 - 错误日志 - NetBeans配置tomcat出错情况总结 -- 部署错误: 启动 Tomcat 失败。-- '127.0.0.1' 不是内部或外部命令，也不是可运行的程序

    真的管用,不知道为啥管用.转载自:http://blog.sina.com.cn/s/blog_709548200102vgy4.html 问题描述: 新安装的NetBeans8.0.2,安装过程中还 ...

  5. SSIS 包部署错误 0xC0010014

    SSIS 包部署错误 0xC0010014 Reinhard 在部署 SSIS 包时,提示如下错误. 由于错误 0xC0010014"发生了一个或多个错误.在此消息之前应有更为具体的错误消息 ...

  6. Hadoop 部署文档

    Hadoop 部署文档 1 先决条件 2 下载二进制文件 3 修改配置文件 3.1 core-site.xml 3.2 hdfs-site.xml 3.3 mapred-site.xml 3.4 ya ...

  7. hadoop进阶----hadoop经验(一)-----生产环境hadoop部署在超大内存服务器的虚拟机集群上vs几个内存较小的物理机

    生产环境 hadoop部署在超大内存服务器的虚拟机集群上 好 还是  几个内存较小的物理机上好? 虚拟机集群优点 虚拟化会带来一些其他方面的功能. 资源隔离.有些集群是专用的,比如给你三台设备只跑一个 ...

  8. Hadoop部署方式-完全分布式(Fully-Distributed Mode)

    Hadoop部署方式-完全分布式(Fully-Distributed Mode) 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 本博客搭建的虚拟机是伪分布式环境(https://w ...

  9. Hadoop部署方式-伪分布式(Pseudo-Distributed Mode)

    Hadoop部署方式-伪分布式(Pseudo-Distributed Mode) 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.下载相应的jdk和Hadoop安装包 JDK:h ...

随机推荐

  1. Asp.net vNext 学习3

    Asp.net vNext 学习之路(三) asp.net vNext 对于构建asp.net 程序带来了一些重大的改变,让我们开发asp.net 程序的时候更加的方便和高效. 1,可以很容易的去管理 ...

  2. [译]Java 设计模式之桥接

    (文章翻译自Java Design Pattern: Bridge) 简单来说,桥梁设计模式是一个两层的抽象. 桥接模式就是从一个抽象中实现中解耦以便两个都可以独立的改变.桥接使用封装聚合而且使用继承 ...

  3. 【从0开始Tornado网站】主页登录和显示的最新文章

    日志首页只能放置在它,这里的美,该<form>使用bootstrap的form-inline修改后的类,例如以下列方式: 前台代码例如以下: {%extends 'main.html'%} ...

  4. 架构师Jack专访：全面认识软件测试架构师

    ◇ 测试架构师的职责 测试的职业通道基本是管理线和技术线两条路. 管理线主要的职责:更多是项目管理和资源管理. 技术线主要的职责:更多是技术管理和业务知识. 软件测试架构师更多就是技术线的带头人.管理 ...

  5. 读书笔记—CLR via C#异常和状态管理

    前言 这本书这几年零零散散读过两三遍了,作为经典书籍,应该重复读反复读,既然我现在开始写博了,我也准备把以前觉得经典的好书重读细读一遍,并且将笔记整理到博客中,好记性不如烂笔头,同时也在写的过程中也可 ...

  6. 【转】 Android用于提示等待的ProgressDialog

    原文地址:http://blog.csdn.net/wleing/article/details/6086321 为了安抚用户等待的焦急心情,我们用ProgressDialog.它的用法书上有例子,就 ...

  7. sql材料分级统计及汇总案例参考

    --第一步:根据系统编号.列.单价分组求和 select CLBH,DJ,sum(SL) as SL,sum(JE) as JE,Lie into #TempSZCMX from #ShouZhiCu ...

  8. beanutils中WrapDynaBean

    public class Emp   { private String  firstName="李";    private String lastName;    public ...

  9. 数据分析R语言1

    数据分析R语言 无意中发现网上的一个数据分析R应用教程,看了几集感觉还不错,本文做一个学习笔记(知识点来源:视频内容+R实战+自己的理解),视频详细的信息请参考http://www.itao521.c ...

  10. AutoPostBack通过现象看本质

    在做人事档案管理系统时遇到一个功能需要实现前台数据(实时)与后台进行交互,解决这个问题首先想到的是应用控件的AutoPostBack属性.本以为这个问题就这样解决了(不用javascript.jque ...