报错栈:

-- ::, INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: = AND =
-- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Working on split: = AND =
-- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: select "EXTEND3","EXTEND2","EXTEND1","MEMO","OPER_DATE","OPER_CODE","FILE_CONTENT","FILE_NAME","INPATIENT_NO","ID" from HIS_SDZL."MDT_FILE" tbl where ( = ) AND ( = )
-- ::, INFO [Thread-] org.apache.sqoop.mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
-- ::, FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:)
at java.lang.StringBuffer.append(StringBuffer.java:)
at java.util.regex.Matcher.appendReplacement(Matcher.java:)
at java.util.regex.Matcher.replaceAll(Matcher.java:)
at java.lang.String.replaceAll(String.java:)
at QueryResult.readFields(QueryResult.java:)
at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:)
at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:)
at org.apache.hadoop.mapred.YarnChild$.run(YarnChild.java:)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:)

调小fetchsize参数也不能解决,那问题很可能是某行数据占用空间很大。根据Sqoop生成的导入表对应的实例化类QueryResult.java的244行可定位到报错列是FILE_CONTENT,是个二进制列, 然后查询原库,果然最大的列长达到180M:

ps: 怎么用标准的sql语句查询 blob字段的大小?
blob字段有好多种。如果是9i的简单的blob字段则应该是 length,或者lengthb也可。实在不行可以用 dbms_lob.getlength()

排查sqoop报错:Error running child : java.lang.OutOfMemoryError: Java heap space的更多相关文章

  1. 解决sqoop报错:java.lang.OutOfMemoryError: Java heap space

    报错栈: -- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: = ) AND ( = ) ...

  2. 【Xamarin报错】 COMPILETODALVIK : UNEXPECTED TOP-LEVEL error java.lang.OutOfMemoryError: Java heap space

    Xamarin Android 编译报错: COMPILETODALVIK : UNEXPECTED TOP-LEVEL error java.lang.OutOfMemoryError: Java ...

  3. 执行Hive出现Error running child : java.lang.OutOfMemoryError: Java heap space错误

    具体错误日志如下: 2018-05-11 15:16:49,429 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running chi ...

  4. 【IntellJ IDEA】idea启动测试类报错Error running 'Test1.test': Command line is too long. Shorten command line for Test1.test or also for JUnit default configuration.

    idea启动测试类报错 Error running 'Test1.test': Command line is too long. Shorten command line for Test1.tes ...

  5. 运行springboot项目报错 Error running 'ResourceApplication': Command line is too long. Shorten comma

    方法1 IDEA 运行报错:Error running '***': Command line is too long 技术标签: IDEA     Error running 'Test': Com ...

  6. Tomcat 启动项目报错 java.lang.OutOfMemoryError: Java heap space

    近日使用myeclipse 部署web项目,启动tomcat时报错: SEVERE: Error waiting for multi-thread deployment of directories ...

  7. Linux运行Java出现“Exception in thread "main" java.lang.OutOfMemoryError: Java heap space”报错

    在运行如下程序时出现“Exception in thread "main" java.lang.OutOfMemoryError: Java heap space”报错: java ...

  8. mycat 报错 java.lang.OutOfMemoryError: Java heap space

    今天排查mysql的错误日志发现  wrapper.log  中有如下错误日志 INFO   | jvm 1    | 2019/10/20 12:52:31 | java.lang.OutOfMem ...

  9. eclipse内存溢出报错:java.lang.OutOfMemoryError:Java heap space

    今天执行了一个比較大的程序,处理的数据达到126MB数据,将数据导入数据库中,用eclipse 来訪问时候,总是出现java.lang.OutOfMemoryError:Java heap space ...

随机推荐

  1. 使用Python快速查询所有指定匹配KEY的办法

    import redis redis_ip = '10.10.14.224' redis_port = 18890 # 配置redis的连接办法 # http://blog.csdn.net/u010 ...

  2. ssh客户端乱码

    export LC_ALL=zh_CN.GB2312;export LANG=zh_CN.GB2312

  3. Tornado入门资料整理

    预备知识 没学过计网的苦逼找点现成一些的东西看吧…… <Restful Web Services>,<HTTP The Definitive Guide>,各种RFC WSGI ...

  4. 大理石在哪儿(UVa10474)

    题目具体描述见:https://uva.onlinejudge.org/index.php?option=com_onlinejudge&Itemid=8&category=835&a ...

  5. Yii 简明学习教程

    Yii是一个基于组件的高性能PHP框架,用于快速开发Web应用程序(下面内容基于Yii 1.1) 1. 典型的工作流 用户向入口脚本index.php发起请求 入口脚本加载应用配置config.php ...

  6. Java中的覆盖和隐藏以及final关键字

    Java覆盖和隐藏 (1) 变量只能被隐藏(包括静态和非静态),不能被覆盖: (2) 可以用子类的静态变量隐藏父类的静态变量,也可以用子类的非静态变量隐藏父类的静态变量,也可以用非最终变量(final ...

  7. windows关闭aslr办法

    关闭aslr方便调试分析. 转:https://www.52pojie.cn/thread-377450-1-1.html windows关闭aslr办法 如 @Hmily  前辈所说, Window ...

  8. Hibernate 对象的生命周期及CRUD操作

    对象状态及生命周期 1.Transient(瞬时状态):new的对象为瞬时态 session中没有数据,数据库中没有数据. 2.Persistent(持久状态):被session管理的对象为持久状态, ...

  9. hdu 4819 二维线段树模板

    /* HDU 4819 Mosaic 题意:查询某个矩形内的最大最小值, 修改矩形内某点的值为该矩形(Mi+MA)/2; 二维线段树模板: 区间最值,单点更新. */ #include<bits ...

  10. [GCJ2017R3]Cooclement

    题目大意: 一种数列按照如下方式变化: 新数列第i位等于原数中数字i的出现次数. 变化过程中数列长度不变. 例如数列12的变化过程为12-11-20-01-10. 现在告诉你一个数列x,请求出x可能是 ...