hadoop常见错误集锦

1  Hadoop namenode -format后Incompatible namespaceIDS 错误解决办法

重新格式话namenode后，会重新分配一个namespaceId, 这个namespaceID就会和存储在datanode上的namespaceID不一致，

解决办法：

a  修改datanode中的namespaceID, 在“dfs.data.dir”/current/VERSION中进行修改，新的namespaceID可以在log中寻找；

b  删除datanode,  dfs.data.dir中的文件，或者更改dfs.data.dir目录为新目录　　

至于如何恢复系统的数据可以参考该文章： http://www.cnblogs.com/Richardzhu/p/3435989.html

2  　SafeModeException

error: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /home/feixiang/hadoop-0.20.2/mapred/system. Name node is in safe mode.
The ratio of reported blocks 0.0000 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /home/feixiang/hadoop-0.20.2/mapred/system. Name node is in safe mode.
The ratio of reported blocks 0.0000 has not reached the threshold 0.9990. Safe mode will be turned off automatically.

原因：namenode每次启动的时候，会进入安全模式，接受datanode上报块数，如果上报的块数损失率达到某个数值（默认是1-0.9990），会一直处于只读不能写的安全模式。namenode每次启动的时候为什么要删除mapred/system文件？？？

有两种方式离开安全模式：

a  在hdfs-site.xml中， 修改dfs.safemode.threshold.pct的值，但是具体为多少，要看log里面的ratio of reported blocks

b  强制离开。命令： hadoop dfsadmin -safemode leave

参考：

http://www.linuxidc.com/Linux/2012-12/76782.htm

http://blog.csdn.net/rzhzhz/article/details/7056793

　

3

2014-03-28 21:15:26,118 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 9003, call addBlock(/home/kanglishan/hadoop/hadoop-1.2.1/mapred/system/jobtracker.info, DFSClient_NONMAPREDUCE_-690631423_1, null) from 127.0.0.1:36367: error: java.io.IOException: File /home/kanglishan/hadoop/hadoop-1.2.1/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /home/kanglishan/hadoop/hadoop-1.2.1/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)
at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

mapred文件夹下没有建立system目录，namenode -format再启动就会出现这个问题，建立这个目录后，重新格式话，再启动就ok,不知道什么原因。