hdfs向hbase上传数据报错分析
通过hbse的import工具向hbase导入文件时出现出错误：
hbase org.apache.hadoop.hbase.mapreduce.Driver import hbase_rgrid_kpi_400 /tmp/dumphbase/20180717/datahbase/datahbase/hbase_rgrid_kpi_400
日志如下：
[BEGIN] 2018/7/19 13:32:31
[9euser@cq-t-h-node0-0-150 ~]$ hbase org.apache.hadoop.hbase.mapreduce.Driver import hbase_rgrid_kpi_100 /tmp/dumphbase/20180717/datahbase/datahbase/hbase_rgrid_kpi_100
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
18/07/19 13:32:12 INFO client.RMProxy: Connecting to ResourceManager at cq-t-h-node0-0-150/192.168.0.150:8032
18/07/19 13:32:13 INFO input.FileInputFormat: Total input paths to process : 25
18/07/19 13:32:13 INFO mapreduce.JobSubmitter: number of splits:25
18/07/19 13:32:13 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1531962677618_0032
18/07/19 13:32:14 INFO impl.YarnClientImpl: Submitted application application_1531962677618_0032
18/07/19 13:32:14 INFO mapreduce.Job: The url to track the job: http://cq-t-h-node0-0-150:8088/proxy/application_1531962677618_0032/
18/07/19 13:32:14 INFO mapreduce.Job: Running job: job_1531962677618_0032
18/07/19 13:32:18 INFO mapreduce.Job: Job job_1531962677618_0032 running in uber mode : false
18/07/19 13:32:18 INFO mapreduce.Job: map 0% reduce 0%
18/07/19 13:32:24 INFO mapreduce.Job: map 8% reduce 0%
18/07/19 13:32:25 INFO mapreduce.Job: map 16% reduce 0%
18/07/19 13:32:26 INFO mapreduce.Job: map 20% reduce 0%
18/07/19 13:32:27 INFO mapreduce.Job: map 32% reduce 0%
18/07/19 13:32:28 INFO mapreduce.Job: map 48% reduce 0%
18/07/19 13:32:29 INFO mapreduce.Job: map 52% reduce 0%
18/07/19 13:32:30 INFO mapreduce.Job: map 60% reduce 0%
18/07/19 13:32:30 INFO mapreduce.Job: Task Id : attempt_1531962677618_0032_m_000004_0, Status : FAILED
', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:760)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:715)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2148)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2182)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
: 4583 times,
at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:258)
at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$2000(AsyncProcess.java:238)
at org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1810)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:240)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:146)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:113)
at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:138)
at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:94)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:664)
at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at org.apache.hadoop.hbase.mapreduce.Import$Importer.processKV(Import.java:210)
at org.apache.hadoop.hbase.mapreduce.Import$Importer.writeResult(Import.java:165)
at org.apache.hadoop.hbase.mapreduce.Import$Importer.map(Import.java:150)
at org.apache.hadoop.hbase.mapreduce.Import$Importer.map(Import.java:133)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 18/07/19 13:32:30 INFO mapreduce.Job: Task Id : attempt_1531962677618_0032_m_000008_0, Status : FAILED
', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:760)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:715)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2148)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2182)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
: 4597 times,
at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:258)
at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$2000(AsyncProcess.java:238)
at org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1810)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:240)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:146)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:113)
at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:138)
at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:94)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:664)
at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at org.apache.hadoop.hbase.mapreduce.Import$Importer.processKV(Import.java:210)
at org.apache.hadoop.hbase.mapreduce.Import$Importer.writeResult(Import.java:165)
at org.apache.hadoop.hbase.mapreduce.Import$Importer.map(Import.java:150)
at org.apache.hadoop.hbase.mapreduce.Import$Importer.map(Import.java:133)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
分析结果是因为 hbase相应的表中没有指定的列簇所致，所以改变了一下hbase表的列簇
hdfs向hbase上传数据报错分析的更多相关文章
- 两个由于php.ini配置错误导致的报错:ajax图片上传报错和exec报错
遇到了两个由于php.ini配置错误导致的报错:ajax图片上传报错和exec报错 首先第一个: 在做一个用ajax图片上传的功能中,php报了这样一个错误:File upload error - u ...
- 解决springMVC文件上传报错： The current request is not a multipart request
转自:https://blog.csdn.net/HaHa_Sir/article/details/79131607 解决springMVC文件上传报错: The current request is ...
- HTTP的上传文件实例分析
这个是http文件传输的一种格式,当时不知道这种格式,废弃. HTTP的上传文件实例分析 由于论坛不支持Word写文章发帖. 首先就是附件发送怎么搞,这个必须解决.论坛是php的.我用Chrome类浏 ...
- 17-Java-文件上传报错(commons-fileupload包和commons-io包不支持JDK版本：UnsupportedClassVersionError: org/apache/commons/io/IOUtils : Unsupported major.minor version 52.0）
文件上传报错(commons-fileupload包和commons-io包不支持JDK版本) 这个bug可把我弄惨了!!!我代码是想通过写个文件上传,我写的文件上传需要用到commons-fileu ...
- Eclipse 上传 删除 下载 分析 hdfs 上的文件
本篇讲解如何通过Eclipse 编写代码去操作分析hdfs 上的文件. 1.在eclipse 下新建Map/Reduce Project项目.如图: 项目建好后,会默认加载一系列相应的jar包. 下 ...
- jquery插件--ajaxfileupload.js上传文件原理分析
英文注解应该是原作者写的吧~说实话,有些if判断里的东西我也没太弄明白,但是大致思路还是OK的. jQuery.extend({ createUploadIframe: function (id, u ...
- 文件上传漏洞靶场分析 UPLOAD_LABS
文件上传漏洞靶场(作者前言) 文件上传漏洞 产生原理 PASS 1) function checkFile() { var file = document.getElementsByName('upl ...
- kindeditor本地上传报错，只限初学者
困扰了我三天的问题,话说百度真的害死人啊,百度上有说路劲错了的,有说包没导的,有说还要改plugins里面的文件的!其实这个都不用动,也有说服务器问题的,还有说缓存的,还有说是ecplise的,反正我 ...
- windows下IIS+PHP解决大文件上传500错问题
linux下改到iis+php后,上传大于2M就出500错,改了php.ini中的upload_max_filesize也不行,最后解决如下: 第一步:修改php.ini 上传大小限制 (以上传500 ...
随机推荐
- ORA-00600:internal error code,arguments:[keltnfy-idmlnit],[46],[1],[],[],[],[],[]
如图:在DBCA进行到3%时.报错 由于/etc/hosts与/etc/sysconfig/network不正确应,所以报错 [root@ocm2 ~]# cat /etc/hosts # Do no ...
- 内核调试工具——strace
简介 strace常用来跟踪进程执行时的系统调用和所接收的信号. 在Linux世界,进程不能直接访问硬件设备,当进程需要访问硬件设备(比如读取磁盘文件,接收网络数据等等)时,必须由用户态模式切换至内核 ...
- The number of sections contained in the collection view after the update (1) must be equal to the number of sections contained in the collection view before the update (0), plus or minus the number of
现象:当删除CollectionView 当中的某个section的时候,报上面的错误 初步分析:当前CollectionView删除前后都不止一个Section,怎么会报那样的错误:猜想可能是相册界 ...
- Java中的引用：强引用、软引用、弱引用、幻象引用（虚引用）
Java语言中,除了原始数据类型的变量(八大基本数据类型),其他都是引用类型,指向各种不同的对象. 理解引用对于我们掌握Java对象生命周期和JVM内部相关机制都是有帮助的. 不同的应用类型,不同之处 ...
- JSON定义及应用
1 什么是JSON? JSON 指的是 JavaScript 对象表示法(JavaScript Object Notation) 是轻量级的文本数据交换格式,JSON 比 XML 更小.更快,更易解析 ...
- ABAP术语-Update Key
Update Key 原文:http://www.cnblogs.com/qiangsheng/archive/2008/03/20/1114171.html Unique character str ...
- MySQL学习【第七篇索引管理及执行计划】
一.索引介绍 1.什么是索引? 索引由如字典,目的就是为了更快寻找到要找的内容. 令搜索查询的数据更有目的性,从而提高数据检索的能力 2.索引类型介绍 1.BTREE: B+树索引 2.HASH: H ...
- [ERROR] Can't find error-message file '/data/mysql/share/errmsg.sys'. Check error-message file location and 'lc-messages-dir' configuration directive.
1. MySQL5.7.21启动时报错: [ERROR] Can't find error-message file '/data/mysql/3307/share/errmsg.sys'. Chec ...
- ORACLE->SQL*Loader[20180712]
https://docs.oracle.com/cd/B28359_01/server.111/b28319/ldr_concepts.htm#g1013706 SQL*Loader将外部 ...
- Shell学习——子shell操作记录转储
概述 主要介绍子shell历史操作记录的保存以及解析,比如python, scala等,用于(准)实时监控用户行为. 背景 一级shell的历史操作记录已由系统实现,当用户从开始登录shell(这里指 ...