Windows7系统运行hadoop报Failed to locate the winutils binary in the hadoop binary path错误
程序运行的过程中,报Failed to locate the winutils binary in the hadoop binary path Java.io.IOException: Could not locate executable\bin\winutils.exe in the Hadoop binaries.
通过断点调试、查看源码发现程序需要根据HADOOP_HOME找到winutils.exe,由于win机器并没有配置该环境变量,所以程序报 null\bin\winutils.exe。
private static String checkHadoopHome() {
// first check the Dflag hadoop.home.dir with JVM scope
String home = System.getProperty("hadoop.home.dir");
// fall back to the system/user-global env variableif (home == null) {
home = System.getenv("HADOOP_HOME");
}
try {
// couldn't find either setting for hadoop's home directoryif (home == null) {
throw new IOException("HADOOP_HOME or hadoop.home.dir are not set.");
}
if (home.startsWith("\"") && home.endsWith("\"")) {
home = home.substring(, home.length()-);
}
// check that the home setting is actually a directory that exists
File homedir = new File(home);
if (!homedir.isAbsolute() || !homedir.exists() || !homedir.isDirectory()) {
thrownew IOException("Hadoop home directory " + homedir
+ " does not exist, is not a directory, or is not an absolute path.");
}
home = homedir.getCanonicalPath();
} catch (IOException ioe) {
if (LOG.isDebugEnabled()) {
LOG.debug("Failed to detect a valid hadoop home directory", ioe);
}
home = null;
}
return home;
}
private static String HADOOP_HOME_DIR = checkHadoopHome();
public static final String getQualifiedBinPath(String executable)
throws IOException {
// construct hadoop bin path to the specified executable
String fullExeName = HADOOP_HOME_DIR + File.separator + "bin"
+ File.separator + executable;
File exeFile = new File(fullExeName);
if (!exeFile.exists()) {
throw new IOException("Could not locate executable " + fullExeName
+ " in the Hadoop binaries.");
}
return exeFile.getCanonicalPath();
}
/** a Windows utility to emulate Unix commands */publicstaticfinal String WINUTILS = getWinUtilsPath();
public static final String getWinUtilsPath() {
String winUtilsPath = null;
try {
if (WINDOWS) {
winUtilsPath = getQualifiedBinPath("winutils.exe");
}
} catch (IOException ioe) {
LOG.error("Failed to locate the winutils binary in the hadoop binary path",
ioe);
}
return winUtilsPath;
}
找到原因后就去网上问了度娘,找到了解决方案,很简单,如下:
1.下载winutils的windows版本
GitHub上,有人提供了winutils的windows的版本,项目地址是:https://github.com/srccodes/hadoop-common-2.2.0-bin,直接下载此项目的zip包,下载后是文件名是hadoop-common-2.2.0-bin-master.zip,随便解压到一个目录
2.配置环境变量
增加用户变量HADOOP_HOME,值是下载的zip包解压的目录,然后在系统变量path里增加$HADOOP_HOME\bin 即可。
再次运行程序,正常执行。
Windows7系统运行hadoop报Failed to locate the winutils binary in the hadoop binary path错误的更多相关文章
- WIN7下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path
之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报Failed to ...
- Spark报错:Failed to locate the winutils binary in the hadoop binary path
之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报 Failed to ...
- Hadoop:开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
问题: windows开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary ...
- Windows本地运行调试Spark或Hadoop程序失败:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path
报错内容 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOExce ...
- windows本地调试安装hadoop(idea) : ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path
1,本地安装hadoop https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/ 下载hadoop对应版本 (我本意是想下载hadoop ...
- Spark- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
运行 mport org.apache.log4j.{Level, Logger} import org.apache.spark.rdd.RDD import org.apache.spark.{S ...
- idea 提示:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException解决方法
Windows系统中的IDEA链接Linux里面的Hadoop的api时出现的问题 提示:ERROR util.Shell: Failed to locate the winutils binary ...
- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
文章发自:http://www.cnblogs.com/hark0623/p/4170172.html 转发请注明 14/12/17 19:18:53 ERROR Shell: Failed to ...
- ERROR [org.apache.hadoop.util.Shell] - Failed to locate the winutils binary in the hadoop binary path
错误日志如下: -- ::, DEBUG [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field org.apache.hadoo ...
随机推荐
- Visual Studio 2013 离线版msdn下载和安装
Visual Studio 2013出来后,并没有自带msdn安装包,而变成了在线安装msdn,好处是msdn可以随时进行更新,坏处是难道以后每次重新安装系统,都需要重新下载吗,如何解决这个问题呢?本 ...
- javascript与 ios通讯解决办法
阔别1年半之久,一个JavaScript和ios通讯的想法终于被实现了(我不知道别人有没有早就实现过~). 记得早期ios内嵌html做通讯时,貌似做好的办法只能是 ios通过url来截取页面发送消息 ...
- spring 事物管理没起到作用
今天在做项目的时候发现配置的spring 事物管理没起到作用.可是配置又是依据官网配置的,不可能会错.最后发现使mysql的问题 普通情况下,mysql会默认提供多种存储引擎,你能够通过以下的查看: ...
- react实现的点击拖拽元素效果
之前用vue做日程管理组件的时候,用到了点击拖拽的效果,即点击元素,鼠标移动到哪里,元素移动到哪里,鼠标松开,拖拽停止,现在在弄react,于是也在想实现这个效果,经过一番折腾,效果出来了,代码如下: ...
- iOS 使用AFN 进行单图和多图上传 摄像头/相册获取图片,压缩图片
图片上传时必要将图片进行压缩,不然会上传失败 首先是同系统相册选择图片和视频.iOS系统自带有UIImagePickerController,可以选择或拍摄图片视频,但是最大的问题是只支持单选,由于项 ...
- MUI 分享功能(微信、QQ 、朋友圈)
配置文件:manifest.json plus ->plugins 下边 "share": {/*配置应用使用分享功能,参考http://ask.dcloud.net.cn/ ...
- with操作符损耗性能的原因
当函数运行是,创建一个[[scope]]指向的被称为作用域链的可变对象集合,作用域链的最前端是一个包含所有的局部变量.参数.this等的被称为“激活对象”的对象. 在标示符查找的过程中,从作用域的最前 ...
- ubuntu 创建桌面快捷方式
$sudo apt-get install gnome-panel $gnome-desktop-item-edit /home/xxx/桌面 --create-new 命令行:填入程序名称,如/u ...
- Fluent Nhibernate Mapping for Sql Views
Views are mapped the same way tables are mapped except that you should put Readonly() in the mapping ...
- 【BZOJ2300】[HAOI2011]防线修建 set维护凸包
[BZOJ2300][HAOI2011]防线修建 Description 近来A国和B国的矛盾激化,为了预防不测,A国准备修建一条长长的防线,当然修建防线的话,肯定要把需要保护的城市修在防线内部了.可 ...