程序运行的过程中,报Failed to locate the winutils binary in the hadoop binary path  Java.io.IOException: Could not locate executable\bin\winutils.exe in the Hadoop binaries.

通过断点调试、查看源码发现程序需要根据HADOOP_HOME找到winutils.exe,由于win机器并没有配置该环境变量,所以程序报 null\bin\winutils.exe。

private static String checkHadoopHome() {
// first check the Dflag hadoop.home.dir with JVM scope
String home = System.getProperty("hadoop.home.dir"); // fall back to the system/user-global env variableif (home == null) {
home = System.getenv("HADOOP_HOME");
}
try {
// couldn't find either setting for hadoop's home directoryif (home == null) {
throw new IOException("HADOOP_HOME or hadoop.home.dir are not set.");
}
if (home.startsWith("\"") && home.endsWith("\"")) {
home = home.substring(, home.length()-);
}
// check that the home setting is actually a directory that exists
File homedir = new File(home);
if (!homedir.isAbsolute() || !homedir.exists() || !homedir.isDirectory()) {
thrownew IOException("Hadoop home directory " + homedir
+ " does not exist, is not a directory, or is not an absolute path.");
}
home = homedir.getCanonicalPath();
} catch (IOException ioe) {
if (LOG.isDebugEnabled()) {
LOG.debug("Failed to detect a valid hadoop home directory", ioe);
}
home = null;
}
return home;
} private static String HADOOP_HOME_DIR = checkHadoopHome(); public static final String getQualifiedBinPath(String executable)
throws IOException {
// construct hadoop bin path to the specified executable
String fullExeName = HADOOP_HOME_DIR + File.separator + "bin"
+ File.separator + executable; File exeFile = new File(fullExeName);
if (!exeFile.exists()) {
throw new IOException("Could not locate executable " + fullExeName
+ " in the Hadoop binaries.");
}
return exeFile.getCanonicalPath();
} /** a Windows utility to emulate Unix commands */publicstaticfinal String WINUTILS = getWinUtilsPath(); public static final String getWinUtilsPath() {
String winUtilsPath = null; try {
if (WINDOWS) {
winUtilsPath = getQualifiedBinPath("winutils.exe");
}
} catch (IOException ioe) {
LOG.error("Failed to locate the winutils binary in the hadoop binary path",
ioe);
} return winUtilsPath;
}

找到原因后就去网上问了度娘,找到了解决方案,很简单,如下:

  1.下载winutils的windows版本

  GitHub上,有人提供了winutils的windows的版本,项目地址是:https://github.com/srccodes/hadoop-common-2.2.0-bin,直接下载此项目的zip包,下载后是文件名是hadoop-common-2.2.0-bin-master.zip,随便解压到一个目录

  2.配置环境变量

  增加用户变量HADOOP_HOME,值是下载的zip包解压的目录,然后在系统变量path里增加$HADOOP_HOME\bin 即可。  

  再次运行程序,正常执行。

Windows7系统运行hadoop报Failed to locate the winutils binary in the hadoop binary path错误的更多相关文章

  1. WIN7下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path

    之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报Failed to ...

  2. Spark报错:Failed to locate the winutils binary in the hadoop binary path

    之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报 Failed to ...

  3. Hadoop:开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

    问题: windows开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary ...

  4. Windows本地运行调试Spark或Hadoop程序失败:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path

    报错内容 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOExce ...

  5. windows本地调试安装hadoop(idea) : ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path

    1,本地安装hadoop https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/ 下载hadoop对应版本 (我本意是想下载hadoop ...

  6. Spark- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

    运行 mport org.apache.log4j.{Level, Logger} import org.apache.spark.rdd.RDD import org.apache.spark.{S ...

  7. idea 提示:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException解决方法

    Windows系统中的IDEA链接Linux里面的Hadoop的api时出现的问题 提示:ERROR util.Shell: Failed to locate the winutils binary ...

  8. ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

    文章发自:http://www.cnblogs.com/hark0623/p/4170172.html  转发请注明 14/12/17 19:18:53 ERROR Shell: Failed to ...

  9. ERROR [org.apache.hadoop.util.Shell] - Failed to locate the winutils binary in the hadoop binary path

    错误日志如下: -- ::, DEBUG [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field org.apache.hadoo ...

随机推荐

  1. Matlab中imread函数使用报错“不应为MATLAB 表达式”分析

    问题描述: 使用imread读取特定路径下的文件时,会提示出错! >> mytest错误: 文件:mytest.m 行:10 列:87不应为 MATLAB 表达式. 出错行: Images ...

  2. VC++ : GetIDsOfNames 调用失败,获取的dispid为-1

    今天调试自己的程序,需要调用一个COM组件中的方法,利用GetIDsOfNames获取调用函数对象的DISPID. GetIDsOfNames: 把COM接口的方法名字和参数(可选)映射成一组DISP ...

  3. nginx 杂谈

    http://blog.sina.com.cn/s/articlelist_1834459124_0_1.html

  4. MAC快捷键使用大全

    当写到标题的时候.突然想起刚接触电脑的时候总是会有一些window希统快捷键大全.foxmail快捷键使用大全,最没想到的是,作为一个说不上资深.也是有了年头的程序猿,到现在.我还要在写一个快捷键使用 ...

  5. Java集合----概述、Collection接口、Iterator接口

    Java 集合概述 Java 集合就像一种容器,可以把多个对象的引用放入容器中. Java 集合类可以用于存储数量不等的多个对象,还可用于保存具有映射关系的关联数组 Java 集合可分为 Set.Li ...

  6. docker学习-docker安装

    win10之外的系统:https://www.docker.com/products/docker-toolbox win10系统:        https://www.docker.com/pro ...

  7. 怎样安装Linux操作系统

    linux是免费的.其实这里的免费只是说linux的内核免费.在linux内核的基础上而产生了众多linux的版本.Linux的发行版说简单点就是将Linux内核与应用软件做一个打包.较知名的发行版有 ...

  8. ios Instruments 内存泄露

    本文转载至 http://my.oschina.net/sunqichao/blog?disp=2&p=3 虽然iOS 5.0版本之后加入了ARC机制,由于相互引用关系比较复杂时,内存泄露还是 ...

  9. JDBC连接oracle地址出错

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 3 ...

  10. poj_2441 状态压缩dp

    题目大意 N头牛,M个谷仓,每个牛c都有它喜欢的若干个谷仓,现在要将这N头牛安排进谷仓,使得每个牛都位于它喜欢的谷仓,而每个谷仓只能有一头牛.求安排的方案总数.N, M <= 20 题目分析 将 ...