Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://192.168.25.128:9000/export/yang/log.1
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
at hadoop1.WordCount.main(WordCount.java:53)

当本人在运行,Hadoop集群自带的wordcount实例的时候,报错内容为输入路径不存在,在网上找了很久没有解决,最后发现是因为我创建的log.1是在本地创建的,并没有上传到hdfs集群中,所以在运行的时候会报错,解决的办法是:执行命令:

[root@master ~]# hadoop fs -put log.1 / #(将log.1文件上传到/目录下)

操作之后可以再次运行命令:

[root@master ~]# hadoop jar /export/servers/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount /1.log /result

执行结果如下:

File System Counters
FILE: Number of bytes read=312
FILE: Number of bytes written=237571
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=300
HDFS: Number of bytes written=206
HDFS: Number of read operations=6
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=1
Launched reduce tasks=1
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=7544
Total time spent by all reduces in occupied slots (ms)=5156
Total time spent by all map tasks (ms)=7544
Total time spent by all reduce tasks (ms)=5156
Total vcore-milliseconds taken by all map tasks=7544
Total vcore-milliseconds taken by all reduce tasks=5156
Total megabyte-milliseconds taken by all map tasks=7725056
Total megabyte-milliseconds taken by all reduce tasks=5279744
Map-Reduce Framework
Map input records=1
Map output records=35
Map output bytes=342
Map output materialized bytes=312
Input split bytes=97
Combine input records=35
Combine output records=25
Reduce input groups=25
Reduce shuffle bytes=312
Reduce input records=25
Reduce output records=25
Spilled Records=50
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=230
CPU time spent (ms)=2110
Physical memory (bytes) snapshot=306843648
Virtual memory (bytes) snapshot=4163534848
Total committed heap usage (bytes)=142278656
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=203
File Output Format Counters
Bytes Written=206

运行成功!

原文链接:https://blog.csdn.net/yangmingtia/article/details/83993206

wordcount报错:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist:的更多相关文章

  1. 报错org.apache.hadoop.mapreduce.lib.input.FileSplit cannot be cast to org.apache.hadoop.mapred.FileSplit

    报错 java.lang.Exception: java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.FileSpli ...

  2. 用java运行Hadoop程序报错:org.apache.hadoop.fs.LocalFileSystem cannot be cast to org.apache.

    用java运行Hadoop例程报错:org.apache.hadoop.fs.LocalFileSystem cannot be cast to org.apache.所写代码如下: package ...

  3. sparkOnYarn报错org.apache.hadoop.fs.FSDataInputStream

    Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInpu ...

  4. 配置MapReduce插件时,弹窗报错org/apache/hadoop/eclipse/preferences/MapReducePreferencePage : Unsupported major.minor version 51.0(Hadoop2.7.3集群部署)

    原因: hadoop-eclipse-plugin-2.7.3.jar 编译的jdk版本和eclipse启动使用的jdk版本不一致导致.  解决方案一: 修改myeclipse.ini文件即可解决. ...

  5. org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/input

    原我是这样写的 //输入数据所在的文件目录 FileInputFormat.addInputPath(job, new Path("/input/")); //mapreduce执 ...

  6. Hadoop报错:org.apache.hadoop.security.AccessControlException: Permission denied: user=xxxx

    问题出现原因: 因为远程提交hadoop的任务的情况下如果,没有hadoop 的系统环境变量,就会读取当前主机的用户名,所以Hadoop集群的节点中没有该用户名的权限,所以出现的异常. 解决方法: S ...

  7. hbase报错org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.NullPointerException

    这个错误找了好长时间,日志看半天,找度娘没什么用,再看stackoverflow,上面有几篇相关的帖子,但是没人回答. 看半天,没有cause,只好到idea中去看sortLocalSorts源码,结 ...

  8. 使用eclipse的快捷键自动生成的map或者reduce函数的参数中:“org.apache.hadoop.mapreduce.Reducer.Context context”

    今天在测试mapreduce的程序时,就是简单的去重,对照课本上的程序和自己的程序,唯一不同的就是“org.apache.hadoop.mapreduce.Reducer.Context contex ...

  9. hadoop项目开发运行报错(log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).)

    使用hadoop+myeclipse开发项目是测试运行报错: log4j:WARN No appenders could be found for logger (org.apache.hadoop. ...

随机推荐

  1. CF1292B Aroma's Search 题解

    Content 给定一个坐标系,已知第一个点的坐标为 \((x_0,y_0)\),第 \(i(i>0)\) 个点的坐标满足这样的两个递推式:\(x_i=a_xx_{i-1}+b_x,y_i=a_ ...

  2. InnoDB学习(五)之数据库锁

    InnoDB存储引擎的默认隔离级别事可重复读,MVCC多版本并发控制仅仅解决了快照读情况下的数据隔离,而对于当前读,InnoDB通过锁来进行并发控制. InnoDB锁 本文主要参考了MySQL官方文档 ...

  3. Linux httpd搭建

    Linux 搭建网站 配置网络 1 改对应配置文件 vi /etc/sysconfig/network-scripts/ifcfg-ens32 2 使用命令 nmcli connection add ...

  4. STC8PROG - Linux下的 STC8G STC8H 烧录工具

    STC8PROG - Linux下的 STC8G STC8H 烧录工具 动机 在Linux下用 VSCode + PlatformIO 做开发, 因为VSCode的界面字体代码提示, 以及自定义的类J ...

  5. c++11之all_of 、 any_of 和 none_of 的用法

    0.时刻提醒自己 Note: vector的释放 1.区别 函数 功能 all_of 区间[开始, 结束)中是否所有的元素都满足判断式p,所有的元素都满足条件返回true,否则返回false. any ...

  6. 【LeetCode】238. Product of Array Except Self 解题报告(Python & C++)

    作者: 负雪明烛 id: fuxuemingzhu 个人博客: http://fuxuemingzhu.cn/ 目录 题目描述 解题方法 两次遍历 日期 题目地址:https://leetcode.c ...

  7. The Best Path

    The Best Path Time Limit: 9000/3000 MS (Java/Others)    Memory Limit: 65535/32768 K (Java/Others)Tot ...

  8. DEV GridControl小结。。

    [转]DEV GridControl小结.. 来自:http://www.cnblogs.com/yuerdongni/archive/2012/09/08/2676753.html   1. 如何解 ...

  9. ZOJ 3785:What day is that day?(数论)

    What day is that day? Time Limit: 2 Seconds Memory Limit: 65536 KB It's Saturday today, what day is ...

  10. 遍历 HashMap 的 5 种最佳方式

    使用 Iterator 遍历 HashMap EntrySet 使用 Iterator 遍历 HashMap KeySet 使用 For-each 循环迭代 HashMap 使用 Lambda 表达式 ...