hadoop job -kill 与 yarn application -kii(作业卡了或作业重复提交或MapReduce任务运行到running job卡住)
问题详情


解决办法
[hadoop@master ~]$ hadoop job -kill job_1493782088693_0001
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it. // :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
Killed job job_1493782088693_0001 [hadoop@master ~]$ hadoop job -kill job_1493782088693_0002
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it. // :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
Killed job job_1493782088693_0002 [hadoop@master ~]$ hadoop job -kill job_1493782088693_0003
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it. // :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
Killed job job_1493782088693_0003
有时候上述这样kill做下来,并不管用,得再来
[hadoop@master ~]$ yarn application -kill application_1493782088693_0001
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Application application_1493782088693_0001 has already finished
[hadoop@master ~]$ yarn application -kill application_1493782088693_0002
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Application application_1493782088693_0002 has already finished
[hadoop@master ~]$ yarn application -kill application_1493782088693_0003
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Application application_1493782088693_0003 has already finished
[hadoop@master ~]$

[hadoop@master ~]$ hadoop job -list
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it. // :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
Total jobs:
JobId State StartTime UserName Queue Priority UsedContainers RsvdContainers UsedMem RsvdMem NeededMem AM info
[hadoop@master ~]$
同时,大家要注意,是否是进程的消失?

也会是你的slave1 还是 slave2的进程自动消失了。注意 ,这是个很隐蔽的问题。
重新停止集群,再重新启动集群。
如果还是出现这个问题的话,则


关于这里参数配置的解释,见
Hadoop YARN配置参数剖析(2)—权限与日志聚集相关参数
注意,对master、slave1和slave2都要操作,然后,再[hadoop@master hadoop-2.6.0]$ sbin/stop-all.sh
再[hadoop@master hadoop-2.6.0]$ sbin/start-all.sh即可。



成功!
hadoop job -kill 与 yarn application -kii(作业卡了或作业重复提交或MapReduce任务运行到running job卡住)的更多相关文章
- hadoop job -kill 和 yarn application -kill 区别
hadoop job -kill 调用的是CLI.java里面的job.killJob(); 这里会分几种情况,如果是能查询到状态是RUNNING的话,是直接向AppMaster发送kill请求的.Y ...
- yarn application -kill application_id yarn kill 超时任务脚本
需求:kill 掉yarn上超时的任务,实现不同队列不同超时时间的kill机制,并带有任务名的白名单功能 此为python脚本,可配置crontab使用 # _*_ coding=utf-8 _*_ ...
- Hadoop(七)YARN的资源调度
一.YARN 概述 YARN 是一个资源调度平台,负责为运算程序提供服务器运算资源,相当于一个分布式的操 作系统平台,而 MapReduce 等运算程序则相当于运行于操作系统之上的应用程序 YARN ...
- spark-shell启动报错:Yarn application has already ended! It might have been killed or unable to launch application master
spark-shell不支持yarn cluster,以yarn client方式启动 spark-shell --master=yarn --deploy-mode=client 启动日志,错误信息 ...
- yarn application ID 增长达到10000后
Job, Task, and Task Attempt IDs In Hadoop 2, MapReduce job IDs are generated from YARN application I ...
- spark利用yarn提交任务报:YARN application has exited unexpectedly with state UNDEFINED
spark用yarn提交任务会报ERROR cluster.YarnClientSchedulerBackend: YARN application has exited unexpectedly w ...
- yarn application命令介绍
yarn application 1.-list 列出所有 application 信息 示例:yarn application -list 2.-appStates <Stat ...
- Apache hadoop namenode ha和yarn ha ---HDFS高可用性
HDFS高可用性Hadoop HDFS 的两大问题:NameNode单点:虽然有StandbyNameNode,但是冷备方案,达不到高可用--阶段性的合并edits和fsimage,以缩短集群启动的时 ...
- 【深入浅出 Yarn 架构与实现】3-1 Yarn Application 流程与编写方法
本篇学习 Yarn Application 编写方法,将带你更清楚的了解一个任务是如何提交到 Yarn ,在运行中的交互和任务停止的过程.通过了解整个任务的运行流程,帮你更好的理解 Yarn 运作方式 ...
随机推荐
- Qt5.3.2_vs10_发布时所需DLL的路径
1. ???\Qt5.3.2_vs2010\5.3\msvc2010_opengl\bin 2.
- js代理模式
代理模式的关键是,当客户不方便直接访问一个对象或者不满足需要的时候,提供一个替身对象来控制对这个对象的访问,客户实际上访问的是替身对象.替身对象对请求做出一些处理之后,再把请求转交给本体对象. 代理模 ...
- wpf数据绑定的论述
(1)绑定模式: <Lable x:Name=lab Content={binding UserName} /> <!--binding相当于SetBinding--> Con ...
- socket和多线程编程资料汇集-基础篇
0 基础 CS结构的分析,server端和client的选取. 1 查看端口是否链接 netstat -an|grep portid 2 root用户抓包 tcpdump port -w fn.cap ...
- element-ui中tabs页设置第一个页面不可关闭
element官网中,有说明添加这个属性,默认为false,但是这个是一个单属性,似乎没法赋值. 这个用css就行了嘛,没必要搞得那么复杂,也不需要设置close属性,按照官网的示例文档即可.css设 ...
- LKDBHelper Sqlite操作数据库
首先这里要说明一下,为什么用FMDB而不用Core Data呢,因为我们不知道Core Data是如何映射对象里面的属性关系的,如果我们更改了属性的话,就会报错 首先是创建LKDBHelper对象 L ...
- MPEG4、XVID、AVC有什么区别
MPEG-4包含XviD和DivX,而AVC优于二者mpeg4 DVD用的多,101mpeg4有AVC格式(加强版MP4)AVC/H.264是一种最新且技术含量最高的视频编码格式,由MPEG-4标准进 ...
- !heap 和 _HEAP_ENTRY
WinDBG提供了!heap命令帮助我们查找heap,同时我们也可以通过dt和MS SYMBOL来了解memory layout. 假设我们有下面一个小程序. int _tmain(int argc, ...
- Linux终端录屏与播放 script 命令
本文由Suzzz原创,发布于 http://www.cnblogs.com/Suzzz/p/4107700.html ,转载请保留此声明. 有时候可能想要记录在终端的所有操作包括输出等,将来作为视频播 ...
- 第一章计算机网络和因特网-day02
1.互联网中的时延:处理时延.排队时延.传输时延.传播时延. 处理时延:检查分组首部和决定该分组导向何处的时间. 排队时延:分组在链路上等待传输的时延. 传输时延:分组经过路由器与交换机的过程的时延. ...