oozie.action.hadoop.LauncherException: IO error Connection timed out: no further information

本文主要针对使用CDH平台的HUE时候碰到两类问题，最终问题并没有得到很好的解决，只是提供了一种绕行方式，欢迎知道的朋友补充。

## **NO 1: HUE执行jar包**

> 第一种报错

org.apache.oozie.action.hadoop.LauncherException: IO error
at org.apache.oozie.action.hadoop.LauncherAM.setRecoveryId(LauncherAM.java:487)
at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:403)
at org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)
at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
at org.apache.oozie.action.hadoop.LauncherAM$1$.run(LauncherAM.java:153)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141)
Caused by: java.lang.NumberFormatException: For input string: "30s"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1267)
at org.apache.hadoop.hdfs.DFSClient$Conf.<init>(DFSClient.java:491)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:620)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:604)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:148)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2598)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2632)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2614)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.oozie.action.hadoop.HdfsOperations.fileExists(HdfsOperations.java:76)
at org.apache.oozie.action.hadoop.LauncherAM.setRecoveryId(LauncherAM.java:473)
... 12 more

> 第二种报错

java.net.ConnectException: Connection timed out: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3090)
at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:778)
at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:693)
at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:354)
at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:617)
at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:841)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:889)
at java.io.DataInputStream.read(DataInputStream.java:100)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:78)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:52)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:112)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:366)
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:265)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
at org.apache.hadoop.yarn.util.FSDownload$2$.run(FSDownload.java:357)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2020-10-14 10:37:32,204 WARN [org.apache.hadoop.hdfs.DFSClient] - Failed to connect to /172.18.0.116:9866 for block, add to deadNodes and continue. java.net.ConnectException: Connection timed out: no further information
java.net.ConnectException: Connection timed out: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3090)
at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:778)
at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:693)
at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:354)
at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:617)
at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:841)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:889)
at java.io.DataInputStream.read(DataInputStream.java:100)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:78)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:52)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:112)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:366)
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:265)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
at org.apache.hadoop.yarn.util.FSDownload$2$.run(FSDownload.java:357)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

> 第三种报错

org.apache.oozie.action.ActionExecutorException: JA009: cache file (mapreduce.job.cache.files) scheme: "hdfs" host: "bdpcdh6301" port: 8020 file: "/user/hue/oozie/workspaces/hue-oozie-1602643504.47/lib/xxxxx_wordcount.jar" conflicts with cache file (mapreduce.job.cache.files) hdfs://bdpcdh6301:8020/bdp/code/wordcount/xxxx_wordcount.jar#xxxxx_wordcount.jar
at org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:469)
at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:447)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1104)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1590)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:243)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:68)
at org.apache.oozie.command.XCommand.call(XCommand.java:291)
at org.apache.oozie.service.CallableQueueServiceCompositeCallable.call(CallableQueueService.java:363)
at org.apache.oozie.service.CallableQueueServiceCompositeCallable.call(CallableQueueService.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.oozie.service.CallableQueueServiceCallableWrapper.run(CallableQueueService.java:210)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.mapred.InvalidJobConfException: cache file (mapreduce.job.cache.files) scheme: "hdfs" host: "xxxxxxxxx" port: 8020 file: "/user/hue/oozie/workspaces/hue-oozie-1602643504.47/lib/xxxxxxxx.jar" conflicts with cache file (mapreduce.job.cache.files) hdfs://bdpcdh6301:8020/bdp/code/wordcount/xxxx_wordcount.jar#xxxxx_wordcount.jar
at org.apache.hadoop.mapreduce.v2.util.LocalResourceBuilder.createLocalResources(LocalResourceBuilder.java:149)
at org.apache.hadoop.mapreduce.v2.util.MRApps.setupDistributedCache(MRApps.java:502)
at org.apache.oozie.action.hadoop.JavaActionExecutor.createAppSubmissionContext(JavaActionExecutor.java:1180)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1091)
... 11 more

```sql
背景：执行mapreduce任务的时候一直报各种错误，上网找了方法，解决了一个错误又出现另外一个错误

原因：说实话到现在也不太清楚

解决方案：使用shell脚本调用本地的jar包

```

## NO 2 : HUE执行shell脚本

```sql
背景：用hue执行shell脚本，单独执行shell脚本没有问题，但是用工作流执行的时候报错

原因：安装任务中用到的hadoop-hdfs依赖与安装的版本不匹配所致

解决方案：因为之前运行过mr，然后改成shell。就一直报这个错。我重新建立了一个工作流，用来执行shell就没问题了

```

oozie.action.hadoop.LauncherException: IO error Connection timed out: no further information的更多相关文章

解决sqoop抽数报错：IO Error: Connection reset
遇到的问题:进行sqoop抽数时,虽然能成功执行,但是过程中有很多这样的信息 19/11/20 15:17:11 INFO mapreduce.Job: Task Id : attempt_15737 ...
项目的ip地址更改，用git从远程提取代码出现错误，提示为 network error connection timed out
昨天公司的ip进行了修改,在今天从远程提取代码的过程中提示network error connection timed out错误,从网上看了一下解决方法 1:打开项目文件夹,点击查看 2:勾选隐藏的 ...
Android Studio Error:Connection timed out: connect.解决方案
遇到了这样的错误: Error:Connection timed out: connect. If you are behind an HTTP proxy, please configure the ...
在linux上通过JDBC连接ORACLE 时总是出现 java.sql.SQLRecoverableException: IO Error: Connection reset 的问题
如标题所述,经过和度娘沟通了不知道多少遍最后看一篇文章解决, 我是通过执行jar来操作DB的.所以在命令中加了一个参数( 加粗,红色) eg: $JAVA_EXE -Djava.security.eg ...
MobaXterm之取消自动关闭连接 Network error :Connection timed out
连一会就出现连接重来的问题配置如下.
[ORACLE]java.sql.SQLRecoverableException: IO Error: Connection rese
随机数引起的阻塞问题程序通过 java -jar -Djava.security.egd=file:/dev/./urandom xxx 的方式执行, http://hongjiang.info/j ...
Putty或MobaXTerm无法连接VMware虚拟机报Network error: Connection timed out的解决方案
当出现无法连接的问题时, 我们要先对可能出现的问题进行梳理, 然后进行排查, 以下我先整理一些可能出现问题的地方: 1. 通过 ping 查看两台终端是否均有联网 windows下通过控制台 cmd ...
java.net.ConnectException: Connection timed out: no further information
ping IP 地址检查是否连上重启虚拟机检查主机
亚马逊的PuTTY连接AWS出现network error connection refused，终极解决方案。
使用PuTTY连接AWS的时候,一直出现network error connection refused.百度了这个问题,大家都说是SSH要设置成22.但是我已经设置过了,为什么还是遇到这个问题呢? ...

随机推荐

Robotframework自动化6-基础关键字介绍3
这一章节介绍一下断言时用到的关键字,断言是写测试用例的必备,没有断言的测试用例是没有灵魂的. 一:Should Be Equal Should Be Equal 是用来判断实践结果和预期结果是否一致 ...
令人困惑的strtotime
经常会有人被strtotime结合-1 month, +1 month, next month的时候搞得很困惑, 然后就会觉得这个函数有点不那么靠谱, 动不动就出问题. 用的时候就会很慌… 这不, 刚 ...
2017年PHP程序员未来路在何方(转)
PHP 从诞生到现在已经有20多年历史,从Web时代兴起到移动互联网退潮,互联网领域各种编程语言和技术层出不穷, Node.js . GO . Python 不断地在挑战 PHP 的地位.这些技术的推 ...
线程的阻塞 sleep() wait() yield()
为了解决对共享存储区的访问冲突,Java 引入了同步机制,现在让我们来考察多个线程对共享资源的访问,显然同步机制已经不够了,因为在任意时刻所要求的资源不一定已经准备好了被访问,反过来,同一时刻准备好了 ...
关于Mybaits
mybatis 返回多表多字段用 mybatis 返回多表多字段用 resultType=”java.util.Map”轻松解决问题.不用加什么DTO.这样前端要什么字段就返回什么字段.不用在对多余的 ...
趣图：调试bug进行时
扩展阅读趣图:大神写实,左脚程序继续运行,右脚程序调试趣图:Bug 多了,总有一个会把你坑了趣图:领导在旁,只求代码无Bug
Python-用装饰器实现递归剪枝
求一个共有10个台阶的楼梯,从下走到上面,一次只能迈出1~3个台阶,并且不能后退,有多少中方法? 上台阶问题逻辑整理: 每次迈出都是 1~3 个台阶,剩下就是 7~9 个台阶如果迈出1个台阶,需要求 ...
C++库文件解析(conio.h)
转载:https://blog.csdn.net/ykmzy/article/details/51276596 Conio.h 控制台输入输出库该文内容部分参照百度百科 Conio.h 在C stan ...
xor 和 or 有什么区别
参考:https://zhidao.baidu.com/question/67532331.html 1.定义区别: ①OR是或运算,A OR B的结果:当A.B中只要有一个或者两个都为1时,结果为1 ...
Ubuntu18.04修改apt-get源
1)备份源文件: sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak 2)查看版本信息如是Linux Mint等Ubuntu衍生版,执行: ...

oozie.action.hadoop.LauncherException: IO error Connection timed out: no further information

oozie.action.hadoop.LauncherException: IO error Connection timed out: no further information的更多相关文章

随机推荐

热门专题