Windows7系统运行hadoop报Failed to locate the winutils binary in the hadoop binary path错误
程序运行的过程中,报Failed to locate the winutils binary in the hadoop binary path Java.io.IOException: Could not locate executable\bin\winutils.exe in the Hadoop binaries.
通过断点调试、查看源码发现程序需要根据HADOOP_HOME找到winutils.exe,由于win机器并没有配置该环境变量,所以程序报 null\bin\winutils.exe。
private static String checkHadoopHome() {
// first check the Dflag hadoop.home.dir with JVM scope
String home = System.getProperty("hadoop.home.dir");
// fall back to the system/user-global env variableif (home == null) {
home = System.getenv("HADOOP_HOME");
}
try {
// couldn't find either setting for hadoop's home directoryif (home == null) {
throw new IOException("HADOOP_HOME or hadoop.home.dir are not set.");
}
if (home.startsWith("\"") && home.endsWith("\"")) {
home = home.substring(, home.length()-);
}
// check that the home setting is actually a directory that exists
File homedir = new File(home);
if (!homedir.isAbsolute() || !homedir.exists() || !homedir.isDirectory()) {
thrownew IOException("Hadoop home directory " + homedir
+ " does not exist, is not a directory, or is not an absolute path.");
}
home = homedir.getCanonicalPath();
} catch (IOException ioe) {
if (LOG.isDebugEnabled()) {
LOG.debug("Failed to detect a valid hadoop home directory", ioe);
}
home = null;
}
return home;
}
private static String HADOOP_HOME_DIR = checkHadoopHome();
public static final String getQualifiedBinPath(String executable)
throws IOException {
// construct hadoop bin path to the specified executable
String fullExeName = HADOOP_HOME_DIR + File.separator + "bin"
+ File.separator + executable;
File exeFile = new File(fullExeName);
if (!exeFile.exists()) {
throw new IOException("Could not locate executable " + fullExeName
+ " in the Hadoop binaries.");
}
return exeFile.getCanonicalPath();
}
/** a Windows utility to emulate Unix commands */publicstaticfinal String WINUTILS = getWinUtilsPath();
public static final String getWinUtilsPath() {
String winUtilsPath = null;
try {
if (WINDOWS) {
winUtilsPath = getQualifiedBinPath("winutils.exe");
}
} catch (IOException ioe) {
LOG.error("Failed to locate the winutils binary in the hadoop binary path",
ioe);
}
return winUtilsPath;
}
找到原因后就去网上问了度娘,找到了解决方案,很简单,如下:
1.下载winutils的windows版本
GitHub上,有人提供了winutils的windows的版本,项目地址是:https://github.com/srccodes/hadoop-common-2.2.0-bin,直接下载此项目的zip包,下载后是文件名是hadoop-common-2.2.0-bin-master.zip,随便解压到一个目录
2.配置环境变量
增加用户变量HADOOP_HOME,值是下载的zip包解压的目录,然后在系统变量path里增加$HADOOP_HOME\bin 即可。
再次运行程序,正常执行。
Windows7系统运行hadoop报Failed to locate the winutils binary in the hadoop binary path错误的更多相关文章
- WIN7下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path
之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报Failed to ...
- Spark报错:Failed to locate the winutils binary in the hadoop binary path
之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报 Failed to ...
- Hadoop:开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
问题: windows开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary ...
- Windows本地运行调试Spark或Hadoop程序失败:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path
报错内容 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOExce ...
- windows本地调试安装hadoop(idea) : ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path
1,本地安装hadoop https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/ 下载hadoop对应版本 (我本意是想下载hadoop ...
- Spark- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
运行 mport org.apache.log4j.{Level, Logger} import org.apache.spark.rdd.RDD import org.apache.spark.{S ...
- idea 提示:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException解决方法
Windows系统中的IDEA链接Linux里面的Hadoop的api时出现的问题 提示:ERROR util.Shell: Failed to locate the winutils binary ...
- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
文章发自:http://www.cnblogs.com/hark0623/p/4170172.html 转发请注明 14/12/17 19:18:53 ERROR Shell: Failed to ...
- ERROR [org.apache.hadoop.util.Shell] - Failed to locate the winutils binary in the hadoop binary path
错误日志如下: -- ::, DEBUG [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field org.apache.hadoo ...
随机推荐
- 下次不用找了,all language code
语言 ID 语言 ID 决定网站中网页文本(例如“网站设置”页上的文本)使用的语言.创建网站时可用的语言取决于在服务器或服务器场中安装的语言模板包.基于 Windows SharePoint Serv ...
- mysql通过mysql_install_db初始化数据目录时使用--user选项的作用是什么?
需求描述: mysql数据库通过mysql_install_db初始化数据目录时,使用了--user选项,这里记录下该参数的作用 参数解释: 1.--user的作用:就是以哪个操作系统用户来执行mys ...
- JavaScript 中的执行上下文和调用栈是什么?
http://zcfy.cc/article/what-is-the-execution-context-amp-stack-in-javascript-by-david-shariff-4007.h ...
- mybatis由浅入深day02_9.3.5使用生成的代码_9.4逆向工程注意事项
9.3.5 使用生成的代码 需要将生成工程中所生成的代码拷贝到自己的工程中. 拷这4个到我们原来的spring_mybatis1216工程下 ItemsMapper.java package cn.i ...
- 工作流JBPM_day01:5-管理流程定义3点改进
工作流JBPM_day01:5-管理流程定义3点改进 1.打包多个文件上传 再部署一个 查询所有看看,旧版本也查出来了 2.查询时只查询所有最新的版本 3.删除指定名称指定key的所有的它的版本
- onTouch
OnTouchOmOnTouchListenerOnTouchEvent View的事件分发 : 对于事件分发机制,举个简单的例子,在一个Activity中只有一个按钮,如果我们想给这个按钮注册 ...
- 复习前面一个月的学习C#感觉道路好艰难啊
今天是复习前面学习的内容,感觉这一个月来真的学习了很多,但是掌握的不好,好多都是在老师讲完课后做起来练习感觉这知识用起来蛮轻松地,但是经过昨天和今天的复习发现好多还是给忘记啦,甚是失落啊,刚开始就知道 ...
- poj_1743 后缀数组
题目大意 给定一串数字,长度为N.定义数字中的某个连续的子串为一个"theme",只要子串满足: (1)长度 >= 5 (2)和该子串相同或者该子串的“变种串”在整串数字中出 ...
- memcached与redis实现的对比
版权声明:本文由田京昆原创文章,转载请注明出处: 文章原文链接:https://www.qcloud.com/community/article/129 来源:腾云阁 https://www.qclo ...
- apply的调用 http://bbs.51js.com/thread-82017-1-3.html
applay 这个是函数Function的方法为什么Array 也能调用这个函数?比如 function a (){ var kk = []; kk = ...