报错栈:

-- ::, INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: = AND =
-- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Working on split: = AND =
-- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: select "EXTEND3","EXTEND2","EXTEND1","MEMO","OPER_DATE","OPER_CODE","FILE_CONTENT","FILE_NAME","INPATIENT_NO","ID" from HIS_SDZL."MDT_FILE" tbl where ( = ) AND ( = )
-- ::, INFO [Thread-] org.apache.sqoop.mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
-- ::, FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:)
at java.lang.StringBuffer.append(StringBuffer.java:)
at java.util.regex.Matcher.appendReplacement(Matcher.java:)
at java.util.regex.Matcher.replaceAll(Matcher.java:)
at java.lang.String.replaceAll(String.java:)
at QueryResult.readFields(QueryResult.java:)
at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:)
at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:)
at org.apache.hadoop.mapred.YarnChild$.run(YarnChild.java:)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:)

调小fetchsize参数也不能解决,那问题很可能是某行数据占用空间很大。根据Sqoop生成的导入表对应的实例化类QueryResult.java的244行可定位到报错列是FILE_CONTENT,是个二进制列, 然后查询原库,果然最大的列长达到180M:

ps: 怎么用标准的sql语句查询 blob字段的大小?
blob字段有好多种。如果是9i的简单的blob字段则应该是 length,或者lengthb也可。实在不行可以用 dbms_lob.getlength()

排查sqoop报错:Error running child : java.lang.OutOfMemoryError: Java heap space的更多相关文章

  1. 解决sqoop报错:java.lang.OutOfMemoryError: Java heap space

    报错栈: -- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: = ) AND ( = ) ...

  2. 【Xamarin报错】 COMPILETODALVIK : UNEXPECTED TOP-LEVEL error java.lang.OutOfMemoryError: Java heap space

    Xamarin Android 编译报错: COMPILETODALVIK : UNEXPECTED TOP-LEVEL error java.lang.OutOfMemoryError: Java ...

  3. 执行Hive出现Error running child : java.lang.OutOfMemoryError: Java heap space错误

    具体错误日志如下: 2018-05-11 15:16:49,429 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running chi ...

  4. 【IntellJ IDEA】idea启动测试类报错Error running 'Test1.test': Command line is too long. Shorten command line for Test1.test or also for JUnit default configuration.

    idea启动测试类报错 Error running 'Test1.test': Command line is too long. Shorten command line for Test1.tes ...

  5. 运行springboot项目报错 Error running 'ResourceApplication': Command line is too long. Shorten comma

    方法1 IDEA 运行报错:Error running '***': Command line is too long 技术标签: IDEA     Error running 'Test': Com ...

  6. Tomcat 启动项目报错 java.lang.OutOfMemoryError: Java heap space

    近日使用myeclipse 部署web项目,启动tomcat时报错: SEVERE: Error waiting for multi-thread deployment of directories ...

  7. Linux运行Java出现“Exception in thread "main" java.lang.OutOfMemoryError: Java heap space”报错

    在运行如下程序时出现“Exception in thread "main" java.lang.OutOfMemoryError: Java heap space”报错: java ...

  8. mycat 报错 java.lang.OutOfMemoryError: Java heap space

    今天排查mysql的错误日志发现  wrapper.log  中有如下错误日志 INFO   | jvm 1    | 2019/10/20 12:52:31 | java.lang.OutOfMem ...

  9. eclipse内存溢出报错:java.lang.OutOfMemoryError:Java heap space

    今天执行了一个比較大的程序,处理的数据达到126MB数据,将数据导入数据库中,用eclipse 来訪问时候,总是出现java.lang.OutOfMemoryError:Java heap space ...

随机推荐

  1. beego学习笔记(4):开发文档阅读(2)

    bee工具的安装和使用 bee 工具是一个为了协助快速开发 beego 项目而创建的项目,通过 bee 您可以很容易的进行 beego 项目的创建.热编译.开发.测试.和部署. go get gith ...

  2. 《自己动手写docker》之namespace部门实验

    动手写一遍,印象不一样! package main import ( "log" "os" "os/exec" "syscall& ...

  3. centos修改oracle字符集

    1.首先以sysdba的身份登录上去 conn /as sysdba2.关闭数据库shutdown immediate;3.以mount打来数据库,startup mount4.设置session S ...

  4. IEEEXtreme 10.0 - Goldbach's Second Conjecture

    这是 meelo 原创的 IEEEXtreme极限编程大赛题解 Xtreme 10.0 - Goldbach's Second Conjecture 题目来源 第10届IEEE极限编程大赛 https ...

  5. Hadoop(七)YARN的资源调度

    一.YARN 概述 YARN 是一个资源调度平台,负责为运算程序提供服务器运算资源,相当于一个分布式的操 作系统平台,而 MapReduce 等运算程序则相当于运行于操作系统之上的应用程序 YARN ...

  6. Python的hasattr() getattr() setattr() 函数使用方法(简介)

    hasattr(object, name)判断一个对象里面是否有name属性或者name方法,返回BOOL值,有name特性返回True, 否则返回False.需要注意的是name要用括号括起来 1 ...

  7. c++ primer 9 顺序容器

    定义: #include <vector> #include <list> #include <deque> vector<int> svec; lis ...

  8. css绝对居中img

    html: <div id="imgs"> <img src="http://pic.616pic.com/ys_b_img/00/03/60/Kt6Q ...

  9. js对小数的操作

    1.丢弃小数部分,保留整数部分 js:parseInt(7/2) 2.向上取整,有小数就整数部分加1 js: Math.ceil(7/2) 3,四舍五入. js: Math.round(7/2) 4, ...

  10. ref:spring-data-XMLBean XXE复现分析

    ref:https://blog.spoock.com/2018/05/16/cve-2018-1259/ 漏洞信息 看pivotal发布的漏洞信息如下 通过发布的漏洞信息可以知道,漏洞组件是在XML ...