跑MR的时候抛出异常:

java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:)
Caused by: java.io.IOException: Task process exit with nonzero status of .
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:)

这里的code是137,常见的

PipeMapRed.waitOutputThreads(): subprocess failed with code N.....
Task process exit with nonzero status of N......

对应的所有code如下:

"OS error code 1: Operation not permitted"
"OS error code 2: No such file or directory"
"OS error code 3: No such process"
"OS error code 4: Interrupted system call"
"OS error code 5: Input/output error"
"OS error code 6: No such device or address"
"OS error code 7: Argument list too long"
"OS error code 8: Exec format error"
"OS error code 9: Bad file descriptor"
"OS error code 10: No child processes"
"OS error code 11: Resource temporarily unavailable"
"OS error code 12: Cannot allocate memory"
"OS error code 13: Permission denied"
"OS error code 14: Bad address"
"OS error code 15: Block device required"
"OS error code 16: Device or resource busy"
"OS error code 17: File exists"
"OS error code 18: Invalid cross-device link"
"OS error code 19: No such device"
"OS error code 20: Not a directory"
"OS error code 21: Is a directory"
"OS error code 22: Invalid argument"
"OS error code 23: Too many open files in system"
"OS error code 24: Too many open files"
"OS error code 25: Inappropriate ioctl for device"
"OS error code 26: Text file busy"
"OS error code 27: File too large"
"OS error code 28: No space left on device"
"OS error code 29: Illegal seek"
"OS error code 30: Read-only file system"
"OS error code 31: Too many links"
"OS error code 32: Broken pipe"
"OS error code 33: Numerical argument out of domain"
"OS error code 34: Numerical result out of range"
"OS error code 35: Resource deadlock avoided"
"OS error code 36: File name too long"
"OS error code 37: No locks available"
"OS error code 38: Function not implemented"
"OS error code 39: Directory not empty"
"OS error code 40: Too many levels of symbolic links"
"OS error code 42: No message of desired type"
"OS error code 43: Identifier removed"
"OS error code 44: Channel number out of range"
"OS error code 45: Level 2 not synchronized"
"OS error code 46: Level 3 halted"
"OS error code 47: Level 3 reset"
"OS error code 48: Link number out of range"
"OS error code 49: Protocol driver not attached"
"OS error code 50: No CSI structure available"
"OS error code 51: Level 2 halted"
"OS error code 52: Invalid exchange"
"OS error code 53: Invalid request descriptor"
"OS error code 54: Exchange full"
"OS error code 55: No anode"
"OS error code 56: Invalid request code"
"OS error code 57: Invalid slot"
"OS error code 59: Bad font file format"
"OS error code 60: Device not a stream"
"OS error code 61: No data available"
"OS error code 62: Timer expired"
"OS error code 63: Out of streams resources"
"OS error code 64: Machine is not on the network"
"OS error code 65: Package not installed"
"OS error code 66: Object is remote"
"OS error code 67: Link has been severed"
"OS error code 68: Advertise error"
"OS error code 69: Srmount error"
"OS error code 70: Communication error on send"
"OS error code 71: Protocol error"
"OS error code 72: Multihop attempted"
"OS error code 73: RFS specific error"
"OS error code 74: Bad message"
"OS error code 75: Value too large for defined data type"
"OS error code 76: Name not unique on network"
"OS error code 77: File descriptor in bad state"
"OS error code 78: Remote address changed"
"OS error code 79: Can not access a needed shared library"
"OS error code 80: Accessing a corrupted shared library"
"OS error code 81: .lib section in a.out corrupted"
"OS error code 82: Attempting to link in too many shared libraries"
"OS error code 83: Cannot exec a shared library directly"
"OS error code 84: Invalid or incomplete multibyte or wide character"
"OS error code 85: Interrupted system call should be restarted"
"OS error code 86: Streams pipe error"
"OS error code 87: Too many users"
"OS error code 88: Socket operation on non-socket"
"OS error code 89: Destination address required"
"OS error code 90: Message too long"
"OS error code 91: Protocol wrong type for socket"
"OS error code 92: Protocol not available"
"OS error code 93: Protocol not supported"
"OS error code 94: Socket type not supported"
"OS error code 95: Operation not supported"
"OS error code 96: Protocol family not supported"
"OS error code 97: Address family not supported by protocol"
"OS error code 98: Address already in use"
"OS error code 99: Cannot assign requested address"
"OS error code 100: Network is down"
"OS error code 101: Network is unreachable"
"OS error code 102: Network dropped connection on reset"
"OS error code 103: Software caused connection abort"
"OS error code 104: Connection reset by peer"
"OS error code 105: No buffer space available"
"OS error code 106: Transport endpoint is already connected"
"OS error code 107: Transport endpoint is not connected"
"OS error code 108: Cannot send after transport endpoint shutdown"
"OS error code 109: Too many references: cannot splice"
"OS error code 110: Connection timed out"
"OS error code 111: Connection refused"
"OS error code 112: Host is down"
"OS error code 113: No route to host"
"OS error code 114: Operation already in progress"
"OS error code 115: Operation now in progress"
"OS error code 116: Stale NFS file handle"
"OS error code 117: Structure needs cleaning"
"OS error code 118: Not a XENIX named type file"
"OS error code 119: No XENIX semaphores available"
"OS error code 120: Is a named type file"
"OS error code 121: Remote I/O error"
"OS error code 122: Disk quota exceeded"
"OS error code 123: No medium found"
"OS error code 124: Wrong medium type"
"OS error code 125: Operation canceled"
"OS error code 126: Required key not available"
"OS error code 127: Key has expired"
"OS error code 128: Key has been revoked"
"OS error code 129: Key was rejected by service"
"OS error code 130: Owner died"
"OS error code 131: State not recoverable"
"MySQL error code 132: Old database file"
"MySQL error code 133: No record read before update"
"MySQL error code 134: Record was already deleted (or record file crashed)"
"MySQL error code 135: No more room in record file"
"MySQL error code 136: No more room in index file"
"MySQL error code 137: No more records (read after end of file)"
"MySQL error code 138: Unsupported extension used for table"
"MySQL error code 139: Too big row"
"MySQL error code 140: Wrong create options"
"MySQL error code 141: Duplicate unique key or constraint on write or update"
"MySQL error code 142: Unknown character set used"
"MySQL error code 143: Conflicting table definitions in sub-tables of MERGE table"
"MySQL error code 144: Table is crashed and last repair failed"
"MySQL error code 145: Table was marked as crashed and should be repaired"
"MySQL error code 146: Lock timed out; Retry transaction"
"MySQL error code 147: Lock table is full; Restart program with a larger locktable"
"MySQL error code 148: Updates are not allowed under a read only transactions"
"MySQL error code 149: Lock deadlock; Retry transaction"
"MySQL error code 150: Foreign key constraint is incorrectly formed"
"MySQL error code 151: Cannot add a child row"
"MySQL error code 152: Cannot delete a parent row"

查证后时MR的JVM对内存设置过高,查看集群设置,我设置的是mapred.child.java.opts是-Xmx8G,mapred.map.child.java.opts和mapred.reduce.child.java.opts都是-Xmx4G(这两个参数优先级会高于mapred.child.java.opts)。

修改mapred.child.java.opts为-Xmx1G,后边两个参数设置为空,再次执行MR,没问题了。

另外,我建了个QQ群:305994766,希望对大数据、算法研发、系统架构感兴趣的朋友能够加入进来,大家一起学习,共同进步(进群请说明自己的公司-职业-昵称).

参考:

https://cloudcelebrity.wordpress.com/2013/08/21/hadoop-mapreduce-job-failure-with-java-io-ioexception-task-process-exit-with-nonzero-status-of-137/

http://langyu.iteye.com/blog/1171091

hadoop系列 第三坑: Task process exit with nonzero status of 137的更多相关文章

  1. Hadoop:Task process exit with nonzero status of 1 异常

    在运行hadoop程序时经常遇到异常 java.io.IOException: Task process exit with nonzero status of 1.网上很多博文都说是磁盘不够的问题. ...

  2. Hadoop 系列(三)Java API

    Hadoop 系列(三)Java API <dependency> <groupId>org.apache.hadoop</groupId> <artifac ...

  3. Hadoop 系列(三)—— 分布式计算框架 MapReduce

    一.MapReduce概述 Hadoop MapReduce 是一个分布式计算框架,用于编写批处理应用程序.编写好的程序可以提交到 Hadoop 集群上用于并行处理大规模的数据集. MapReduce ...

  4. Hadoop系列(三):hadoop基本测试

    下面是对hadoop的一些基本测试示例 Hadoop自带测试类简单使用 这个测试类名叫做 hadoop-mapreduce-client-jobclient.jar,位置在 hadoop/share/ ...

  5. Hadoop 系列文章(三) 配置部署启动YARN及在YARN上运行MapReduce程序

    这篇文章里我们将用配置 YARN,在 YARN 上运行 MapReduce. 1.修改 yarn-env.sh 环境变量里的 JAVA_HOME 路径 [bamboo@hadoop-senior ha ...

  6. hadoop系列三:mapreduce的使用(一)

    转载请在页首明显处注明作者与出处 http://www.cnblogs.com/zhuxiaojie/p/7224772.html 一:说明 此为大数据系列的一些博文,有空的话会陆续更新,包含大数据的 ...

  7. Python GUI之tkinter窗口视窗教程大集合(看这篇就够了) JAVA日志的前世今生 .NET MVC采用SignalR更新在线用户数 C#多线程编程系列(五)- 使用任务并行库 C#多线程编程系列(三)- 线程同步 C#多线程编程系列(二)- 线程基础 C#多线程编程系列(一)- 简介

    Python GUI之tkinter窗口视窗教程大集合(看这篇就够了) 一.前言 由于本篇文章较长,所以下面给出内容目录方便跳转阅读,当然也可以用博客页面最右侧的文章目录导航栏进行跳转查阅. 一.前言 ...

  8. hadoop系列四:mapreduce的使用(二)

    转载请在页首明显处注明作者与出处 一:说明 此为大数据系列的一些博文,有空的话会陆续更新,包含大数据的一些内容,如hadoop,spark,storm,机器学习等. 当前使用的hadoop版本为2.6 ...

  9. Hadoop 系列(一)基本概念

    Hadoop 系列(一)基本概念 一.Hadoop 简介 Hadoop 是一个由 Apache 基金会所开发的分布式系统基础架构,它可以使用户在不了解分布式底层细节的情況下开发分布式程序,充分利用集群 ...

随机推荐

  1. 分布式文件系统FastDFS安装教程

    前言 FastDFS(Fast Distributed File System)是一款开源轻量级分布式文件系统,本文不讲解原理和架构,只是在个人使用部署过程中耗费了好长时间和精力,遇到了很多的坑,于是 ...

  2. Linux 源码安装 FFmpeg

    加班次数频繁,已经快一年没有写博了.由于此次在项目中使用到了 FFmpeg 来处理音视频,顾记录下在 Linux 系统中源码安装 FFmpeg 以便不时之需. 1. 安装 yasm yasm 是一个汇 ...

  3. HTML5 audio元素如何使用js与jquery控制其事件

    前言: 每一次遇见问题想到的就是怎么解决?最好的方法还是查询网络媒体,更好的办法是让自己记忆,只有自己理解到了才真正是属于自己.要做一个订单提醒功能,没有使用audio相关的插件,虽然插件无数,还是喜 ...

  4. scikit-learn入门导航

    scikit-learn是一个非常强大的机器学习库, 提供了很多常见机器学习算法的实现. scikit-learn可以通过pip进行安装: pip install -U scikit-learn 不过 ...

  5. [PHP] 算法-合并两个有序链表为一个有序链表的PHP实现

    合并两个有序的链表为一个有序的链表: 类似归并排序中合并两个数组的部分 1.遍历链表1和链表2,比较链表1和2中的元素大小 2.如果链表1结点大于链表2的结点,该结点放入第三方链表 3.链表1往下走一 ...

  6. (4)Jquery1.8.3快速入门_基本选择器

    一.Jquery选择器: 基本选择器: 1.id                           #id      根据元素的id获取的唯一元素. 2.class                  ...

  7. eclipse中安装windowbuilder插件、应用及卸载

    一.安装windowbuilder插件 1.找到对应版本的windowbuilder 连接地址:http://www.eclipse.org/windowbuilder/download.php 如图 ...

  8. Vue 系列之 渲染与事件处理

    渲染相关 列表渲染 与 条件渲染 Vue 中的常见的渲染有 列表渲染 和 条件渲染 所谓条件渲染,则是通过添加一定的逻辑条件来进行 Dom 元素的操作 v-if v-else v-else-if &l ...

  9. Immuable详解以及在React中的实战

    转载自:https://zhuanlan.zhihu.com/p/20295971, 今天看到这篇文章后情不自禁的转载过来了,我的天老爷,我看到后直接菊花一紧,这写的太好了,直接写进我心坎里了,我必须 ...

  10. mysql学习目录

    MySQL数据库 mysql 之库, 表的简易操作 mysql之行(记录)的详细操作 mysql之单表查询 mysql之多表查询 Navicat安装及简单使用 mysql之Navicat工具.pymy ...