hadoop的单机部署很简单也不容易出错，但是对生产环境的价值和意义不大，但是可以快速用于开发。

部署hadoop的错误原因不少，并且很奇怪。

比如，用户名不同，造成客户端和服务器通讯产生认证失败的错误，客户端，服务器各节点的用户名应当是一致的，并且个节点应该建立ssh的无认证登陆。

一、出现下面错误：

13/07/09 13:57:07 INFO ipc.Client: Retrying connect to server: master/192.168.2.200:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

或

java.net.ConnectException: Call to master/192.168.2.200:9000 failed on connection exception: java.net.ConnectException: Connection refused

        at org.apache.hadoop.ipc.Client.wrapException(Client.java:1136)

        at org.apache.hadoop.ipc.Client.call(Client.java:1112)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.renewLease(Unknown Source)

        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.renewLease(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:379)

        at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:378)

        at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:400)

        at org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69)

        at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273)

        at java.lang.Thread.run(Thread.java:722)

Caused by: java.net.ConnectException: Connection refused

        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:719)

        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)

        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511)

        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481)

        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:453)

        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:579)

        at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:202)

        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1243)

        at org.apache.hadoop.ipc.Client.call(Client.java:1087)

        ... 14 more

是客户端无法连接服务器造成的，可能是服务器没有启动或者启动了防火墙。

二、出现下面错误：

13/07/09 13:57:36 ERROR hdfs.DFSClient: Failed to close file /tmp/web304069331.log

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web304069331.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

原因很多，namenode节点和datanode节点不能正常通讯造成的。根据查询启动日志看到datanode节点没有办法解析机器名造成的，所以修改/etc/hostname和/etc/hosts文件。

三、出现一下错误：

13/07/09 13:59:01 INFO hdfs.DFSClient: Exception in createBlockOutputStream 192.168.2.201:50010 java.net.SocketTimeoutException: 63000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=/192.168.2.201:50010]

13/07/09 13:59:01 INFO hdfs.DFSClient: Abandoning blk_-6965665250189110825_1679

13/07/09 13:59:01 INFO hdfs.DFSClient: Excluding datanode 192.168.2.201:50010

13/07/09 13:59:01 WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web465901718.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

13/07/09 13:59:01 WARN hdfs.DFSClient: Error Recovery for blk_-6965665250189110825_1679 bad datanode[0] nodes == null

13/07/09 13:59:01 WARN hdfs.DFSClient: Could not get block locations. Source file "/tmp/web465901718.log" - Aborting...

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web465901718.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

^C七月 09, 2013 2:00:46 下午 org.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler ["http-bio-8080"]

七月 09, 2013 2:00:46 下午 org.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler ["ajp-bio-8009"]

七月 09, 2013 2:00:46 下午 org.apache.catalina.core.StandardService stopInternal

INFO: Stopping service Catalina

13/07/09 14:00:46 ERROR hdfs.DFSClient: Failed to close file /tmp/web465901718.log

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web465901718.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

七月 09, 2013 2:00:46 下午 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads

是因为客户端与数据节点通讯失败造成的。客户端程序应该能够和所有的节点通讯才能保证数据的传输正常。

四、hadoop namenode -format时出现：

Format aborted in /home/cnsworder/hadoop/name

删除掉这个文件夹重新格式化。

hadoop部署错误的更多相关文章

  1. Hadoop部署方式-本地模式(Local (Standalone) Mode)

    Hadoop部署方式-本地模式(Local (Standalone) Mode) 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. Hadoop总共有三种运行方式.本地模式(Local ...

  2. hadoop常见错误总结三

    问题导读:1.... could only be replicated to 0 nodes, instead of 1 ...可能的原因是什么?2.Error: java.lang.NullPoin ...

  3. hadoop部署小结的命令

    hadoop部署总结的命令 学习笔记,转自:hadoop部署总结的命令http://www.aboutyun.com/thread-5385-1-1.html(出处: about云开发)

  4. Tomcat学习笔记 - 错误日志 - NetBeans配置tomcat出错情况总结 -- 部署错误: 启动 Tomcat 失败。-- '127.0.0.1' 不是内部或外部命令，也不是可运行的程序

    真的管用,不知道为啥管用.转载自:http://blog.sina.com.cn/s/blog_709548200102vgy4.html 问题描述: 新安装的NetBeans8.0.2,安装过程中还 ...

  5. SSIS 包部署错误 0xC0010014

    SSIS 包部署错误 0xC0010014 Reinhard 在部署 SSIS 包时,提示如下错误. 由于错误 0xC0010014"发生了一个或多个错误.在此消息之前应有更为具体的错误消息 ...

  6. Hadoop 部署文档

    Hadoop 部署文档 1 先决条件 2 下载二进制文件 3 修改配置文件 3.1 core-site.xml 3.2 hdfs-site.xml 3.3 mapred-site.xml 3.4 ya ...

  7. hadoop进阶----hadoop经验(一)-----生产环境hadoop部署在超大内存服务器的虚拟机集群上vs几个内存较小的物理机

    生产环境 hadoop部署在超大内存服务器的虚拟机集群上 好 还是  几个内存较小的物理机上好? 虚拟机集群优点 虚拟化会带来一些其他方面的功能. 资源隔离.有些集群是专用的,比如给你三台设备只跑一个 ...

  8. Hadoop部署方式-完全分布式(Fully-Distributed Mode)

    Hadoop部署方式-完全分布式(Fully-Distributed Mode) 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 本博客搭建的虚拟机是伪分布式环境(https://w ...

  9. Hadoop部署方式-伪分布式(Pseudo-Distributed Mode)

    Hadoop部署方式-伪分布式(Pseudo-Distributed Mode) 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.下载相应的jdk和Hadoop安装包 JDK:h ...

随机推荐

  1. 一道关于CSS选择器优先级的题

    <span style="font-size:14px;"><html> <head> <style type="text/cs ...

  2. C语言语法笔记 – 高级用法 指针数组 指针的指针 二维数组指针 结构体指针 链表 | IT宅.com

    原文:C语言语法笔记 – 高级用法 指针数组 指针的指针 二维数组指针 结构体指针 链表 | IT宅.com C语言语法笔记 – 高级用法 指针数组 指针的指针 二维数组指针 结构体指针 链表 | I ...

  3. 判断文件是否存在，不存在创建文件&&判断文件夹是否存在，不存在创建文件夹

    1.判断文件是否存在,不存在创建文件 File file=new File("C:\\Users\\QPING\\Desktop\\JavaScript\\2.htm"); if( ...

  4. Angular指令（一）

    Angular开发者手册重点翻译之指令(一) 创建自定义的指令 这个文章将解释什么需要在自己的angularjs应用中创建自己的指令,以及如何实现它. 什么是指令 在高的层面上讲,指令是DOM元素中的 ...

  5. jQuery EasyUI API - Layout - Layout[原创汉化官方API]

    最近在学习jQuery EasyUI,发现中文的文档好少,部分文档不错但它是鸟语的,为了大家也为了自己学习吧,汉化做一下笔记. 有没有说清楚的,或者翻译不正确的地方还请大家谅解指出.. 由于工作时间原 ...

  6. OpenWrt的UCI系统

    http://wiki.openwrt.org/doc/uci UCI是Unified Configuration Interface的缩写,翻译成中文就是统一配置接口,用途就是为OpenWrt提供一 ...

  7. leetcode[91] Subsets II

    给定一个数组,返回所有的非重复的可能.例如给定 If S = [1,2,2], a solution is: [ [2], [1], [1,2,2], [2,2], [1,2], [] ] 其实这题类 ...

  8. 由浅入深学习.NET CLR 系列：目录

    经过对Android的一阵折腾,些许熟悉了一些Java的东东,又开始转战.NET.我觉得学习最好和工作不要相离太远,才会更加随笔随意,索性整理一些比较系统的.NET的基础知识学习学习.一提起学习.NE ...

  9. 安装系统提示选中的磁盘具有MBR分区表

    引用:http://jingyan.baidu.com/article/ff42efa9693e88c19e22020c.html 进入bios看到  UEFI状态时  Enabled 启用状态,那么 ...

  10. JobTracker等相关功能模块初始化

    [Hadoop代码笔记]Hadoop作业提交之JobTracker等相关功能模块初始化 一.概要描述 本文重点描述在JobTracker一端接收作业.调度作业等几个模块的初始化工作.想过模块的介绍会在 ...