问题:

将MongoDB数据导入Hive,按照https://blog.csdn.net/thriving_fcl/article/details/51471248文章,在hive建外部表与mongodb做映射后,执行后出现

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON

建表语句如下:

CREATE EXTERNAL TABLE mongotohive
  id string,
  userid string,
  age bigint,
  status string
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","userid":"user_id","age":"age","status":"status"}')
TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/mydb.users'); 

mongodb 数据如下:

db.users.find()
{ "_id" : ObjectId("5b456e33a93daf7ae53e6419"), "user_id" : "abc123", "age" : 58, "status" : "D" }
{ "_id" : ObjectId("5b45705ca93daf7ae53e8b2a"), "user_id" : "bcd001", "age" : 45, "status" : "C" }

解决方案:

将mongo-hadoop-core-2.0.0.jar、mongo-hadoop-hive-2.0.0.jar、mongo-java-driver-3.7.1.jar三个jar包放到hive的lib文件夹下后,再次运行成功。如下:

hive> CREATE EXTERNAL TABLE mongotohive
    > ( 
    >   id string,
    >   userid string,
    >   age bigint,
    >   status string
    > )
    > STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
    > WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","userid":"user_id","age":"age","status":"status"}')
    > TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/mydb.users');
OK
Time taken: 1.431 seconds
hive> select * from mongotohive;
OK
5b456e33a93daf7ae53e6419        abc123  58      D
5b45705ca93daf7ae53e8b2a        bcd001  45      C
Time taken: 0.601 seconds, Fetched: 2 row(s)
hive> 

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON的更多相关文章

  1. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: An exception was thrown while adding/validating class(es) :

    在hive命令行创建表时报错: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. ...

  2. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(me

    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(me ...

  3. Hive的Shell里hive> 执行操作时,出现FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask错误的解决办法(图文详解)

    不多说,直接上干货! 这个问题,得非 你的hive和hbase是不是同样都是CDH版本,还是一个是apache版本,一个是CDH版本. 问题详情 [kfk@bigdata-pro01 apache-h ...

  4. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

    hive启动后,出现以下异常 hive> show databases; FAILED: Error / failed on connection exception: java.net.Con ...

  5. hive_异常_01_(未解决)FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V

    一.如果出现如下错误需要编译源码 需要重新编译Hbase-handler源码 步骤如下: 准备Jar包: 将Hbase 中lib下的jar包和Hive中lib下的jar包全部导入到一起. 记得删除里面 ...

  6. hive遇到FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误

    hive遇到FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误 起因 ...

  7. Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/

    Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/ ...

  8. 解决hiveserver2报错:java.io.IOException: Job status not available - Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

    用户使用的sql: select count( distinct patient_id ) from argus.table_aa000612_641cd8ce_ceff_4ea0_9b27_0a3a ...

  9. java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

    执行Hive查询: Console是这样报错的 java.sql.SQLException: Error from org.apache.hadoop.hive.ql.exec.mr.MapRedTa ...

随机推荐

  1. URL整理

    Airtest project官网 http://airtest.netease.com/ poco辅助文档:http://poco.readthedocs.io/zh_CN/latest/index ...

  2. SpringMVC请求体参数处理问题

    如果请求定义为application/json格式,则要用Spring MVC中@RequestBody参数才能接受(用@RequestParam参数接受会报400错误),但SpringMVC的@Re ...

  3. RCC 和 RTC

    RCC是STM32的时钟控制器,可开启或关闭各总线的时钟,在使用各外设功能必须先开启其对应的时钟,没有这个时钟内部的各器件就不能运行.RTC是STM32内部集成的一个简单的时钟(计时用),如果不用就关 ...

  4. Mongo 用户创建及权限管理

    Mongo版本3.0之前使用的是db.addUser(),但3.0之后使用的是db.createUser() 内建的角色: 数据库用户角色:read.readWrite; 数据库管理角色:dbAdmi ...

  5. mybatis 一对多,(多对一,一对一

    多对一,和一对一是同一种写法,每种写法又分在数据库关联和在mybatis关联 1,多对一,一对一数据库关联 2,多对一,一对一mybatis关联 3,一对多,数据库关联,注意,Java type改of ...

  6. amqp 和 exchange 详细解释

    amqp  的 excange 字面意思是一个交换机.他的任务是吧 消息 分配给消息队列. amqp 的  exchange 有三种,分别是 Direct , fanout 和 toppic.三种. ...

  7. CentOS6.5系统下RPM包安装MySQL5.6(转)

    1.查看操作系统相关信息. [root@linuxidc ~]# cat /etc/issue CentOS release 6.5 (Final) Kernel \r on an \m [root@ ...

  8. HDU3501——欧拉函数裸题

    给整数N(1 ≤ N ≤ 1000000000),求小于N的与N不互素的所有正整数的和. 思路:1.用欧拉函数求出小于N的与N互素的正整数的个数: 2.若 p 与 N 互素,则 N-p 必与 N 互素 ...

  9. 手动增加pe节并修改oep

    一直想学学怎么动动pe文件,学习了几篇文章尤其是寒晨的文章后,自己动手也尝试了一下加节和修改oep,写出来供和我一样菜的一起进步. 一.       增加pe节需要的操作 1.    确定内存中的节的 ...

  10. curl发送post请求,统计响应时间

    curl  -o /dev/null -s -w %{time_namelookup}::%{time_connect}::%{time_starttransfer}::%{time_total}:: ...