程序运行的过程中,报Failed to locate the winutils binary in the hadoop binary path  Java.io.IOException: Could not locate executable\bin\winutils.exe in the Hadoop binaries.

通过断点调试、查看源码发现程序需要根据HADOOP_HOME找到winutils.exe,由于win机器并没有配置该环境变量,所以程序报 null\bin\winutils.exe。

private static String checkHadoopHome() {
// first check the Dflag hadoop.home.dir with JVM scope
String home = System.getProperty("hadoop.home.dir"); // fall back to the system/user-global env variableif (home == null) {
home = System.getenv("HADOOP_HOME");
}
try {
// couldn't find either setting for hadoop's home directoryif (home == null) {
throw new IOException("HADOOP_HOME or hadoop.home.dir are not set.");
}
if (home.startsWith("\"") && home.endsWith("\"")) {
home = home.substring(, home.length()-);
}
// check that the home setting is actually a directory that exists
File homedir = new File(home);
if (!homedir.isAbsolute() || !homedir.exists() || !homedir.isDirectory()) {
thrownew IOException("Hadoop home directory " + homedir
+ " does not exist, is not a directory, or is not an absolute path.");
}
home = homedir.getCanonicalPath();
} catch (IOException ioe) {
if (LOG.isDebugEnabled()) {
LOG.debug("Failed to detect a valid hadoop home directory", ioe);
}
home = null;
}
return home;
} private static String HADOOP_HOME_DIR = checkHadoopHome(); public static final String getQualifiedBinPath(String executable)
throws IOException {
// construct hadoop bin path to the specified executable
String fullExeName = HADOOP_HOME_DIR + File.separator + "bin"
+ File.separator + executable; File exeFile = new File(fullExeName);
if (!exeFile.exists()) {
throw new IOException("Could not locate executable " + fullExeName
+ " in the Hadoop binaries.");
}
return exeFile.getCanonicalPath();
} /** a Windows utility to emulate Unix commands */publicstaticfinal String WINUTILS = getWinUtilsPath(); public static final String getWinUtilsPath() {
String winUtilsPath = null; try {
if (WINDOWS) {
winUtilsPath = getQualifiedBinPath("winutils.exe");
}
} catch (IOException ioe) {
LOG.error("Failed to locate the winutils binary in the hadoop binary path",
ioe);
} return winUtilsPath;
}

找到原因后就去网上问了度娘,找到了解决方案,很简单,如下:

  1.下载winutils的windows版本

  GitHub上,有人提供了winutils的windows的版本,项目地址是:https://github.com/srccodes/hadoop-common-2.2.0-bin,直接下载此项目的zip包,下载后是文件名是hadoop-common-2.2.0-bin-master.zip,随便解压到一个目录

  2.配置环境变量

  增加用户变量HADOOP_HOME,值是下载的zip包解压的目录,然后在系统变量path里增加$HADOOP_HOME\bin 即可。  

  再次运行程序,正常执行。

Windows7系统运行hadoop报Failed to locate the winutils binary in the hadoop binary path错误的更多相关文章

  1. WIN7下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path

    之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报Failed to ...

  2. Spark报错:Failed to locate the winutils binary in the hadoop binary path

    之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报 Failed to ...

  3. Hadoop:开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

    问题: windows开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary ...

  4. Windows本地运行调试Spark或Hadoop程序失败:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path

    报错内容 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOExce ...

  5. windows本地调试安装hadoop(idea) : ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path

    1,本地安装hadoop https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/ 下载hadoop对应版本 (我本意是想下载hadoop ...

  6. Spark- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

    运行 mport org.apache.log4j.{Level, Logger} import org.apache.spark.rdd.RDD import org.apache.spark.{S ...

  7. idea 提示:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException解决方法

    Windows系统中的IDEA链接Linux里面的Hadoop的api时出现的问题 提示:ERROR util.Shell: Failed to locate the winutils binary ...

  8. ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

    文章发自:http://www.cnblogs.com/hark0623/p/4170172.html  转发请注明 14/12/17 19:18:53 ERROR Shell: Failed to ...

  9. ERROR [org.apache.hadoop.util.Shell] - Failed to locate the winutils binary in the hadoop binary path

    错误日志如下: -- ::, DEBUG [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field org.apache.hadoo ...

随机推荐

  1. Visual Studio 2013 离线版msdn下载和安装

    Visual Studio 2013出来后,并没有自带msdn安装包,而变成了在线安装msdn,好处是msdn可以随时进行更新,坏处是难道以后每次重新安装系统,都需要重新下载吗,如何解决这个问题呢?本 ...

  2. javascript与 ios通讯解决办法

    阔别1年半之久,一个JavaScript和ios通讯的想法终于被实现了(我不知道别人有没有早就实现过~). 记得早期ios内嵌html做通讯时,貌似做好的办法只能是 ios通过url来截取页面发送消息 ...

  3. spring 事物管理没起到作用

    今天在做项目的时候发现配置的spring 事物管理没起到作用.可是配置又是依据官网配置的,不可能会错.最后发现使mysql的问题 普通情况下,mysql会默认提供多种存储引擎,你能够通过以下的查看: ...

  4. react实现的点击拖拽元素效果

    之前用vue做日程管理组件的时候,用到了点击拖拽的效果,即点击元素,鼠标移动到哪里,元素移动到哪里,鼠标松开,拖拽停止,现在在弄react,于是也在想实现这个效果,经过一番折腾,效果出来了,代码如下: ...

  5. iOS 使用AFN 进行单图和多图上传 摄像头/相册获取图片,压缩图片

    图片上传时必要将图片进行压缩,不然会上传失败 首先是同系统相册选择图片和视频.iOS系统自带有UIImagePickerController,可以选择或拍摄图片视频,但是最大的问题是只支持单选,由于项 ...

  6. MUI 分享功能(微信、QQ 、朋友圈)

    配置文件:manifest.json plus ->plugins 下边 "share": {/*配置应用使用分享功能,参考http://ask.dcloud.net.cn/ ...

  7. with操作符损耗性能的原因

    当函数运行是,创建一个[[scope]]指向的被称为作用域链的可变对象集合,作用域链的最前端是一个包含所有的局部变量.参数.this等的被称为“激活对象”的对象. 在标示符查找的过程中,从作用域的最前 ...

  8. ubuntu 创建桌面快捷方式

    $sudo apt-get install gnome-panel $gnome-desktop-item-edit  /home/xxx/桌面 --create-new 命令行:填入程序名称,如/u ...

  9. Fluent Nhibernate Mapping for Sql Views

    Views are mapped the same way tables are mapped except that you should put Readonly() in the mapping ...

  10. 【BZOJ2300】[HAOI2011]防线修建 set维护凸包

    [BZOJ2300][HAOI2011]防线修建 Description 近来A国和B国的矛盾激化,为了预防不测,A国准备修建一条长长的防线,当然修建防线的话,肯定要把需要保护的城市修在防线内部了.可 ...