程序运行的过程中,报Failed to locate the winutils binary in the hadoop binary path  Java.io.IOException: Could not locate executable\bin\winutils.exe in the Hadoop binaries.

通过断点调试、查看源码发现程序需要根据HADOOP_HOME找到winutils.exe,由于win机器并没有配置该环境变量,所以程序报 null\bin\winutils.exe。

private static String checkHadoopHome() {
// first check the Dflag hadoop.home.dir with JVM scope
String home = System.getProperty("hadoop.home.dir"); // fall back to the system/user-global env variableif (home == null) {
home = System.getenv("HADOOP_HOME");
}
try {
// couldn't find either setting for hadoop's home directoryif (home == null) {
throw new IOException("HADOOP_HOME or hadoop.home.dir are not set.");
}
if (home.startsWith("\"") && home.endsWith("\"")) {
home = home.substring(, home.length()-);
}
// check that the home setting is actually a directory that exists
File homedir = new File(home);
if (!homedir.isAbsolute() || !homedir.exists() || !homedir.isDirectory()) {
thrownew IOException("Hadoop home directory " + homedir
+ " does not exist, is not a directory, or is not an absolute path.");
}
home = homedir.getCanonicalPath();
} catch (IOException ioe) {
if (LOG.isDebugEnabled()) {
LOG.debug("Failed to detect a valid hadoop home directory", ioe);
}
home = null;
}
return home;
} private static String HADOOP_HOME_DIR = checkHadoopHome(); public static final String getQualifiedBinPath(String executable)
throws IOException {
// construct hadoop bin path to the specified executable
String fullExeName = HADOOP_HOME_DIR + File.separator + "bin"
+ File.separator + executable; File exeFile = new File(fullExeName);
if (!exeFile.exists()) {
throw new IOException("Could not locate executable " + fullExeName
+ " in the Hadoop binaries.");
}
return exeFile.getCanonicalPath();
} /** a Windows utility to emulate Unix commands */publicstaticfinal String WINUTILS = getWinUtilsPath(); public static final String getWinUtilsPath() {
String winUtilsPath = null; try {
if (WINDOWS) {
winUtilsPath = getQualifiedBinPath("winutils.exe");
}
} catch (IOException ioe) {
LOG.error("Failed to locate the winutils binary in the hadoop binary path",
ioe);
} return winUtilsPath;
}

找到原因后就去网上问了度娘,找到了解决方案,很简单,如下:

  1.下载winutils的windows版本

  GitHub上,有人提供了winutils的windows的版本,项目地址是:https://github.com/srccodes/hadoop-common-2.2.0-bin,直接下载此项目的zip包,下载后是文件名是hadoop-common-2.2.0-bin-master.zip,随便解压到一个目录

  2.配置环境变量

  增加用户变量HADOOP_HOME,值是下载的zip包解压的目录,然后在系统变量path里增加$HADOOP_HOME\bin 即可。  

  再次运行程序,正常执行。

Windows7系统运行hadoop报Failed to locate the winutils binary in the hadoop binary path错误的更多相关文章

  1. WIN7下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path

    之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报Failed to ...

  2. Spark报错:Failed to locate the winutils binary in the hadoop binary path

    之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报 Failed to ...

  3. Hadoop:开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

    问题: windows开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary ...

  4. Windows本地运行调试Spark或Hadoop程序失败:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path

    报错内容 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOExce ...

  5. windows本地调试安装hadoop(idea) : ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path

    1,本地安装hadoop https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/ 下载hadoop对应版本 (我本意是想下载hadoop ...

  6. Spark- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

    运行 mport org.apache.log4j.{Level, Logger} import org.apache.spark.rdd.RDD import org.apache.spark.{S ...

  7. idea 提示:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException解决方法

    Windows系统中的IDEA链接Linux里面的Hadoop的api时出现的问题 提示:ERROR util.Shell: Failed to locate the winutils binary ...

  8. ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

    文章发自:http://www.cnblogs.com/hark0623/p/4170172.html  转发请注明 14/12/17 19:18:53 ERROR Shell: Failed to ...

  9. ERROR [org.apache.hadoop.util.Shell] - Failed to locate the winutils binary in the hadoop binary path

    错误日志如下: -- ::, DEBUG [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field org.apache.hadoo ...

随机推荐

  1. 放在github pages上的静态网站怎么取消绑定自定义域名?

    使用GitHub Pages搭建的静态网站绑定了自定义域名操作,但是想反悔怎么办? 删除CNAME文件,重新删除仓库,新建后均不行~ 解决办法:清除一下浏览器缓存,或者用别的浏览器打开就好了,因为之前 ...

  2. sql替换数据库字段中的字符

    UPDATE `table_name` SET `field_name` = replace (`field_name`,'from_str','to_str') WHERE ……说明:table_n ...

  3. HTML的框架结构

    <html> <head> <title>HTML的框架结构</title> </head> <frameset frameborde ...

  4. Java中自己定义缓存方式

    说说自己在开发中经常用到的写法. /** * 数据缓存 * @author * */public class DataCache {    /** 对象缓存*/    public static Ma ...

  5. VS2015编译TIFF3.8.0源码

    没有CMakeLists.txt,不能使用CMake GUI了.源文件中有makefile.vc,所以使用nmake 进入VS2015命令行 nmake -f makefile.vc 修改nmake. ...

  6. 微信红包随机生成算法(PHP版)

    /** * 求一个数的平方 * @param $n */ function sqr($n){ return $n*$n; } /** * 生产min和max之间的随机数,但是概率不是平均的,从min到 ...

  7. JS-运动基础——案例应用:淡入淡出效果

    <!DOCTYPE html> <html> <head> <meta charset="UTF-8"> <title> ...

  8. webpack中,css中打包背景图,路径报错

    css-loader: //打包样式中背景图 { test: /\.(png|jpg)$/, loader: "url-loader?limit=8192&name=images/[ ...

  9. 代码片段,lucene基本操作(基于lucene4.10.2)

    1.最基本的创建索引: @Test public void testIndex(){ try { Directory directory = FSDirectory.open(new File(LUC ...

  10. postgresql数据库创建、修改、删除

    一.数据库创建 语法: Command:     CREATE DATABASEDescription: create a new databaseSyntax:CREATE DATABASE nam ...