Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://192.168.25.128:9000/export/yang/log.1
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
at hadoop1.WordCount.main(WordCount.java:53)

当本人在运行,Hadoop集群自带的wordcount实例的时候,报错内容为输入路径不存在,在网上找了很久没有解决,最后发现是因为我创建的log.1是在本地创建的,并没有上传到hdfs集群中,所以在运行的时候会报错,解决的办法是:执行命令:

[root@master ~]# hadoop fs -put log.1 / #(将log.1文件上传到/目录下)

操作之后可以再次运行命令:

[root@master ~]# hadoop jar /export/servers/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount /1.log /result

执行结果如下:

File System Counters
FILE: Number of bytes read=312
FILE: Number of bytes written=237571
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=300
HDFS: Number of bytes written=206
HDFS: Number of read operations=6
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=1
Launched reduce tasks=1
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=7544
Total time spent by all reduces in occupied slots (ms)=5156
Total time spent by all map tasks (ms)=7544
Total time spent by all reduce tasks (ms)=5156
Total vcore-milliseconds taken by all map tasks=7544
Total vcore-milliseconds taken by all reduce tasks=5156
Total megabyte-milliseconds taken by all map tasks=7725056
Total megabyte-milliseconds taken by all reduce tasks=5279744
Map-Reduce Framework
Map input records=1
Map output records=35
Map output bytes=342
Map output materialized bytes=312
Input split bytes=97
Combine input records=35
Combine output records=25
Reduce input groups=25
Reduce shuffle bytes=312
Reduce input records=25
Reduce output records=25
Spilled Records=50
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=230
CPU time spent (ms)=2110
Physical memory (bytes) snapshot=306843648
Virtual memory (bytes) snapshot=4163534848
Total committed heap usage (bytes)=142278656
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=203
File Output Format Counters
Bytes Written=206

运行成功!

原文链接:https://blog.csdn.net/yangmingtia/article/details/83993206

wordcount报错:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist:的更多相关文章

  1. 报错org.apache.hadoop.mapreduce.lib.input.FileSplit cannot be cast to org.apache.hadoop.mapred.FileSplit

    报错 java.lang.Exception: java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.FileSpli ...

  2. 用java运行Hadoop程序报错:org.apache.hadoop.fs.LocalFileSystem cannot be cast to org.apache.

    用java运行Hadoop例程报错:org.apache.hadoop.fs.LocalFileSystem cannot be cast to org.apache.所写代码如下: package ...

  3. sparkOnYarn报错org.apache.hadoop.fs.FSDataInputStream

    Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInpu ...

  4. 配置MapReduce插件时,弹窗报错org/apache/hadoop/eclipse/preferences/MapReducePreferencePage : Unsupported major.minor version 51.0(Hadoop2.7.3集群部署)

    原因: hadoop-eclipse-plugin-2.7.3.jar 编译的jdk版本和eclipse启动使用的jdk版本不一致导致.  解决方案一: 修改myeclipse.ini文件即可解决. ...

  5. org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/input

    原我是这样写的 //输入数据所在的文件目录 FileInputFormat.addInputPath(job, new Path("/input/")); //mapreduce执 ...

  6. Hadoop报错:org.apache.hadoop.security.AccessControlException: Permission denied: user=xxxx

    问题出现原因: 因为远程提交hadoop的任务的情况下如果,没有hadoop 的系统环境变量,就会读取当前主机的用户名,所以Hadoop集群的节点中没有该用户名的权限,所以出现的异常. 解决方法: S ...

  7. hbase报错org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.NullPointerException

    这个错误找了好长时间,日志看半天,找度娘没什么用,再看stackoverflow,上面有几篇相关的帖子,但是没人回答. 看半天,没有cause,只好到idea中去看sortLocalSorts源码,结 ...

  8. 使用eclipse的快捷键自动生成的map或者reduce函数的参数中:“org.apache.hadoop.mapreduce.Reducer.Context context”

    今天在测试mapreduce的程序时,就是简单的去重,对照课本上的程序和自己的程序,唯一不同的就是“org.apache.hadoop.mapreduce.Reducer.Context contex ...

  9. hadoop项目开发运行报错(log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).)

    使用hadoop+myeclipse开发项目是测试运行报错: log4j:WARN No appenders could be found for logger (org.apache.hadoop. ...

随机推荐

  1. CF808A Lucky Year 题解

    Content 年份中有不超过 \(1\) 个非 \(0\) 数字的年份是幸运年份.现给出当前年份 \(n\),求到下一个幸运年份还要等多久. 数据范围:\(1\leqslant n\leqslant ...

  2. CF667A Pouring Rain 题解

    Content 一个水桶直径为 \(d\) 厘米,初始时水面高度为 \(h\) 厘米.你每秒钟喝 \(v\) 毫升水,而由于下雨,水桶里面的水在不喝水的时候每秒会上升 \(e\) 厘米.求你最少需要多 ...

  3. ffmpeg 系列博客

    https://www.ffmpeg.org/download.html#build-macffmpeg 系列博文https://me.csdn.net/blog/leixiaohua1020http ...

  4. springboot等javaweb项目将jar包安装(打包)到本地Maven仓库

    在开发过程中有时会用到maven仓库里没有的jar包或者本地的jar包 1.打开jar所在文件夹,假设我们要将 taobao-sdk-java-auto_1479188381469-20200121. ...

  5. c++设计模式概述之中介

    代码写的不够规范,目的是为了缩短篇幅,实际中请不要这样做. 1.概述 A.中介模式,主要对象有两类: 中介和用户,类比生活中的房产中介公司,中介手中掌握着用户的资料,当然,用户手中也有中介的联系方式. ...

  6. 【LeetCode】718. Maximum Length of Repeated Subarray 解题报告(Python)

    [LeetCode]718. Maximum Length of Repeated Subarray 解题报告(Python) 标签(空格分隔): LeetCode 作者: 负雪明烛 id: fuxu ...

  7. 基于React和Node.JS的表单录入系统的设计与实现

    一.写在前面 这是一个真实的项目,项目已经过去好久了,虽然很简单,但还是有很多思考点,跟随着笔者的脚步,一起来看看吧.本文纯属虚构,涉及到的相关信息均已做虚构处理, 二.背景 人活着一定要有信仰,没有 ...

  8. 《Head First设计模式》读书笔记

    前言:本文是记录我在阅读<Head First设计模式>这本书时,做得相关笔记,相关示例代码地址:design-patterns.由于本书不是将设计原则和设计模式分开讲述的,而是在讲一个设 ...

  9. iGPT and ViT

    目录 概 主要内容 iGPT ViT 代码 Chen M., Radford A., Child R., Wu J., Jun H., Dhariwal P., Luan D., Sutskever ...

  10. ZFNet: Visualizing and Understanding Convolutional Networks

    目录 论文结构 反卷积 ZFnet的创新点主要是在信号的"恢复"上面,什么样的输入会导致类似的输出,通过这个我们可以了解神经元对输入的敏感程度,比如这个神经元对图片的某一个位置很敏 ...