idea执行mapreduce报错 Could not locate Hadoop executable: C:\hadoop-3.1.1\bin\winutils.exe
window执行mapreduce报错
Exception in thread "main" java.lang.RuntimeException: java.io.FileNotFoundException: Could not locate Hadoop executable: C:\hadoop-3.1.1\bin\winutils.exe -see https://wiki.apache.org/hadoop/WindowsProblems
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:)
at org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:)
at org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:)
at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:)
at org.apache.hadoop.mapreduce.Job$.run(Job.java:)
at org.apache.hadoop.mapreduce.Job$.run(Job.java:)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:)
at com.venn.demo.WordCount.main(WordCount.java:)
Caused by: java.io.FileNotFoundException: Could not locate Hadoop executable: C:\hadoop-3.1.\bin\winutils.exe -see https://wiki.apache.org/hadoop/WindowsProblems
at org.apache.hadoop.util.Shell.getQualifiedBinInner(Shell.java:)
at org.apache.hadoop.util.Shell.getQualifiedBin(Shell.java:)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:)
at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:)
at com.venn.util.HdfsUtil.isPathExist(HdfsUtil.java:)
at com.venn.util.HdfsUtil.handleOutPutPath(HdfsUtil.java:)
at com.venn.demo.WordCount.main(WordCount.java:)
注： C:\hadoop-3.1.1 是配置的 HADOOP_HOME 路径
下载对应版本的 winutils.exe，放到 路径： C:\hadoop-3.1.1\bin\ 下。
然后，你会发现又有个新的bug。O(∩_∩)O哈哈~
Exception in thread "main" java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:640)
at org.apache.hadoop.fs.FileUtil.canRead(FileUtil.java:1223)
at org.apache.hadoop.util.DiskChecker.checkAccessByFileMethods(DiskChecker.java:160)
at org.apache.hadoop.util.DiskChecker.checkDirInternal(DiskChecker.java:100)
at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:77)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:315)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:378)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:152)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:133)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:117)
at org.apache.hadoop.mapred.LocalDistributedCacheManager.setup(LocalDistributedCacheManager.java:124)
at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:172)
at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:788)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:254)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
at com.venn.demo.WordCount.main(WordCount.java:77)
再下载个对应版本的 hadoop.ddl ，放到 C:\Windows\System32 ，就搞定了
一般来说， winutils.exe 和 hadoop.ddl 同一个大版本应该都可以
idea执行mapreduce报错 Could not locate Hadoop executable: C:\hadoop-3.1.1\bin\winutils.exe的更多相关文章
- Could not locate executable E:\SoftWave\Hadoop-2.2.0\bin\winutils.exe in the Hadoop binaries解决办法
需要下载windows版本 bin目录下的文件,替换hadoop目录下原来的bin目录下的文件.下载网址是: https://github.com/srccodes/hadoop-common-2.2 ...
- 执行mysqld_safe报错:mysqld does not exist or is not executable
执行mysqld_safe报错: [root@edu data]# /usr/local/mysql5.7/bin/mysqld_safe --user=mysql160427 12:41:28 my ...
- 数据库执行sql报错Got a packet bigger than 'max_allowed_packet' bytes及重启mysql
准备在mysql上使用数据库A,但mysql5经过重装后,上面的数据库已丢失,只得通过之前备份的A.sql重新生成数据库A. 1.执行sql报错 在执行A.sql的过程中,出现如下错误:Got a p ...
- selenium执行js报错
selenium执行js报错 Traceback (most recent call last): dr.execute_script(js) File "C:\Python27\l ...
- mysql执行update报错1175解决方法
mysql执行update报错 update library set status=true where 1=1 Error Code: 1175. You are using safe update ...
- 转 sqlplus执行sql报错：ORA-01756:
1.sqlplus执行sql报错:ORA-01756: quoted string not properly terminated 分类: 技术 在SQLPLUS中执行SQL文件时 ...
- js执行函数报错Cannot set property 'value' of null怎么解决？
js执行函数报错Cannot set property 'value' of null 的解决方案: 原因:dom还没有完全加载 第一步:所以js建议放在body下面执行, 第二步:window.on ...
- dotnetcore ef 调用多个数据库时用户命令执行操作报错
dotnetcore ef 调用多个数据库时用户命令执行操作报错 1.多个DbContext 时报错: 报错: More than one DbContext was found. Specify w ...
- mysql执行update报错 Err] 1055 - 'information_schema.PROFILING.SEQ' isn't in GROUP BY
mysql执行update报错 Err] 1055 - 'information_schema.PROFILING.SEQ' isn't in GROUP BY 今天开发的同事发来如下错误信息,最最简 ...
随机推荐
- django模型系统（三）
1,自定义主键字段的创建 AutoFiled(pirmary_key=True) # 一般不会自定义 2,order_by asc desc 表关系 OneToOne student = mod ...
- docker 小结
1.docker修改镜像地址: /etc/docker/daemon.json 2.docker 启动容器: docker run -t -i ubuntu:14.04 /bin/bash 3.查看容 ...
- 笔记本 原来win10系统改装win7系统遇到 invaid signature detected.check secure boot policy setup问题
这次操作的笔记本电脑是 华硕R414U 大家如果遇到类似问题的话也可以参考这个方法,但是必须搞清楚电脑的型号,型号不同操作起来有差别的 我这里选择的重装系统的方法是最简单粗暴的硬盘安装方法,怎么硬 ...
- [java,2017-05-04] 创建word文档
package test; import java.text.SimpleDateFormat; import java.util.Date; import com.aspose.words.Data ...
- win10 linux 子系统 所在 目录
C:\Users\用户名\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\r ...
- 61.纯 CSS 创作一只咖啡壶(这个不好看）
原文地址:https://segmentfault.com/a/1190000015376202 感想: 好像不像呀,啊啊啊.伪元素.定位.动画.width和height包括内边距|边框|内容区. H ...
- python 如何把小数变成百分数格式
1. 数据样本 ,valid_rate,homework_rate,inter_rate,playback_rate,zhujiang_good_comment5_rate,fudao_good_co ...
- UI5-学习篇-17-云端WEB IDE开发
1.前提条件 SAP云平台账号已申请 云连接器设置完成:UI5-学习篇-15-云连接SAP Cloud Connector 云平台Destination设置完成:UI5-学习篇-16-云端SCP-De ...
- Kafka自带zookeeper报错INFO Got user-level KeeperException when processing xxx Error Path:/brokers Error:KeeperErrorCode = NodeExists for /brokers (org.apache.zookeeper.server.PrepRequestProcessor)
问题描述: 按照kafka官方文档的操作步骤,解压kafka压缩包后.依次启动zookeeper,和kafka服务 kafka服务启动后,查看到zookeeper日志里有以下异常 问题原因及解决办法: ...
- cdnbest的proxy里api用法案例：
用户的proxy帐号里api key要设置好,那个key设置后是不显示的,但会显示已设置 key是自已随便生成的 $uid = 22222; $skey = 'langansafe&*#'; ...