把本地文件系统,复制到dfs文件系统时报错的错

[hadoop@localhost ~]$ hdfs dfs -copyFromLocal /home/hadoop/mk.txt /xg_test/
03:59:35,556 WARN hdfs.DataStreamer: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /xg_test/mk.txt._COPYING_ could only be written to 0 of the 1 minReplication nodes. There are 0 datanode(s) running and no node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:2116)
at org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.chooseTargetForNewBlock(FSDirWriteFileOp.java:287)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2688)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:875)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:559)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1491)
at org.apache.hadoop.ipc.Client.call(Client.java:1437)
at org.apache.hadoop.ipc.Client.call(Client.java:1347)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy11.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:504)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
at com.sun.proxy.$Proxy12.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSOutputStream.addBlock(DFSOutputStream.java:1078)
at org.apache.hadoop.hdfs.DataStreamer.locateFollowingBlock(DataStreamer.java:1865)
at org.apache.hadoop.hdfs.DataStreamer.nextBlockOutputStream(DataStreamer.java:1668)
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:716)
copyFromLocal: File /xg_test/mk.txt._COPYING_ could only be written to 0 of the 1 minReplication nodes. There are 0 datanode(s) running and no node(s) are excluded in this operation.

这边出现上述问题格式化多次hadoop,导致datanode不正确,决解的办法是:

1、在hadoop数据目录下(安装目录或者自己配置的目录,根据实际情况而定)……/data/dfs/data,把里面的current文件夹删除

2、然后从新执行一下 hadoop namenode -format

3、再使用start-dfs.sh和start-yarn.sh 重启一下hadoop,用jps命令看一下就可以看见datanode启动成功后就可以了

异常笔记:运行hdfs copyFromLocal 上传文件报错的更多相关文章

  1. Linux - xshell上传文件报错乱码

    xshell上传文件报错乱码,解决方法 rz -be 回车 下载sz  filename

  2. Azkban上传文件报错installation Failed.Error chunking

    azkaban 上传文件报错Caused by: java.sql.SQLException: The size of BLOB/TEXT data inserted in one transacti ...

  3. Tomcat上传文件报错:returned a response status of 403 Forbidden

    出现这样的错误是没有权限对服务器进行写操作.需要在这个项目所在的tomcat中配置可写操作即可: 在tomcat的web.xml添加下面代码: <init-param><param- ...

  4. rz上传文件报错:rpm Read Signature failed: sigh blob(1268): BAD, read returned 0

    上传文件报错: [root@www localdisk]# rpm -ivh cobbler* error: cobbler-2.8.4-4.el7.x86_64.rpm: rpm  Read  Si ...

  5. SecureCRT sftp上传文件报错:put: failed to upload xxx 拒绝访问

    1.问题 使用sftp上传文件时报错:put: failed to upload xxx 拒绝访问.类似下图所示: 2.原因 造成这个问题的原因可能有两个,一是要上到的那个目录剩余磁盘空间不足,二是打 ...

  6. 云笔记项目- 上传文件报错"java.lang.IllegalStateException: File has been moved - cannot be read again"

    在做文件上传时,当写入上传的文件到文件时,会报错“java.lang.IllegalStateException: File has been moved - cannot be read again ...

  7. 【大数据系列】hadoop上传文件报错_COPYING_ could only be replicated to 0 nodes

    使用hadoop上传文件 hdfs dfs -put  XXX 17/12/08 17:00:39 WARN hdfs.DFSClient: DataStreamer Exception org.ap ...

  8. springmvc上传文件报错org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.springframework.web.multipart.MultipartFile]

    在用springmvc+mybatis进行项目开发时,上传文件抛异常... org.springframework.beans.BeanInstantiationException: Could no ...

  9. hadoop上传文件报错

    19/06/06 16:09:26 INFO hdfs.DFSClient: Exception in createBlockOutputStream java.io.IOException: Bad ...

随机推荐

  1. 21.拉取&删除远程分支

    拉取 当 git fetch 命令从服务器上抓取本地没有的数据时,它并不会修改工作目录中的内容. 它只会获取数据然后让你自己合并. 然而,有一个命令叫作 git pull 在大多数情况下它的含义是一个 ...

  2. winform 写入和读取TXT文件

    C# winform写入和读取TXT文件 string str; str=this.textBox1.Text; StreamWriter sw = new StreamWriter(Applicat ...

  3. SQL Server 2017 on Linux

    此文转载,原文地址:https://docs.microsoft.com/zh-cn/sql/linux/sql-server-linux-whats-new CTP 2.0 The CTP 2.0 ...

  4. 【深入理解JAVA虚拟机】第三部分.虚拟机执行子系统.3.函数调用与执行

    这章原名叫“虚拟机字节码执行引擎”,实际就是讲的函数如何调用和执行的. 1.概述 “虚拟机”是一个相对于“物理机”的概念,这两种机器都有代码执行能力, 其区别是物理机的执行引擎是直接建立在处理器. 硬 ...

  5. 使用virtualenv的正确姿势

    关于virtualenv是什么我想我不用介绍的吧.用python做开发的人要是不会用virtualenv,总有一天会被各种环境恶心死.那么我今天就来讲一讲使用virtualenv的正确姿势. virt ...

  6. 弃坑pexpect,入坑paramiko

    上文书说到,ssh库pexpect的使用,简直就是个“月亮公主”——满眼全是坑.勉强把程序写好了,跑起来的时候发现了一个新坑,让我不可抗拒的把它弃掉了——经常莫名其妙的连不上服务器!开线程连接14台服 ...

  7. Androidannotation使用之@Rest获取资源及用户登录验证(一)

    版权声明:本文为博主原创文章.未经博主同意不得转载. https://blog.csdn.net/NUPTboyZHB/article/details/24384713 简介: 上一篇博文简单的介绍了 ...

  8. 阅读《C陷阱与缺陷》的知识增量

    版权声明:本文为Focustc原创文章.转载请注明作者及出处. https://blog.csdn.net/caozhankui/article/details/35925939 看完<C陷阱与 ...

  9. BZOJ3940:[USACO]Censoring(AC自动机,栈)

    Description Farmer John has purchased a subscription to Good Hooveskeeping magazine for his cows, so ...

  10. BZOJ1009:[HNOI2008]GT考试(AC自动机,矩乘DP)

    Description 阿申准备报名参加GT考试,准考证号为N位数X1X2....Xn(0<=Xi<=9),他不希望准考证号上出现不吉利的数字. 他的不吉利数学A1A2...Am(0< ...