问题:

将MongoDB数据导入Hive,按照https://blog.csdn.net/thriving_fcl/article/details/51471248文章,在hive建外部表与mongodb做映射后,执行后出现

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON

建表语句如下:

CREATE EXTERNAL TABLE mongotohive
  id string,
  userid string,
  age bigint,
  status string
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","userid":"user_id","age":"age","status":"status"}')
TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/mydb.users'); 

mongodb 数据如下:

db.users.find()
{ "_id" : ObjectId("5b456e33a93daf7ae53e6419"), "user_id" : "abc123", "age" : 58, "status" : "D" }
{ "_id" : ObjectId("5b45705ca93daf7ae53e8b2a"), "user_id" : "bcd001", "age" : 45, "status" : "C" }

解决方案:

将mongo-hadoop-core-2.0.0.jar、mongo-hadoop-hive-2.0.0.jar、mongo-java-driver-3.7.1.jar三个jar包放到hive的lib文件夹下后,再次运行成功。如下:

hive> CREATE EXTERNAL TABLE mongotohive
    > ( 
    >   id string,
    >   userid string,
    >   age bigint,
    >   status string
    > )
    > STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
    > WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","userid":"user_id","age":"age","status":"status"}')
    > TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/mydb.users');
OK
Time taken: 1.431 seconds
hive> select * from mongotohive;
OK
5b456e33a93daf7ae53e6419        abc123  58      D
5b45705ca93daf7ae53e8b2a        bcd001  45      C
Time taken: 0.601 seconds, Fetched: 2 row(s)
hive> 

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON的更多相关文章

  1. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: An exception was thrown while adding/validating class(es) :

    在hive命令行创建表时报错: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. ...

  2. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(me

    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(me ...

  3. Hive的Shell里hive> 执行操作时,出现FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask错误的解决办法(图文详解)

    不多说,直接上干货! 这个问题,得非 你的hive和hbase是不是同样都是CDH版本,还是一个是apache版本,一个是CDH版本. 问题详情 [kfk@bigdata-pro01 apache-h ...

  4. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

    hive启动后,出现以下异常 hive> show databases; FAILED: Error / failed on connection exception: java.net.Con ...

  5. hive_异常_01_(未解决)FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V

    一.如果出现如下错误需要编译源码 需要重新编译Hbase-handler源码 步骤如下: 准备Jar包: 将Hbase 中lib下的jar包和Hive中lib下的jar包全部导入到一起. 记得删除里面 ...

  6. hive遇到FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误

    hive遇到FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误 起因 ...

  7. Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/

    Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/ ...

  8. 解决hiveserver2报错:java.io.IOException: Job status not available - Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

    用户使用的sql: select count( distinct patient_id ) from argus.table_aa000612_641cd8ce_ceff_4ea0_9b27_0a3a ...

  9. java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

    执行Hive查询: Console是这样报错的 java.sql.SQLException: Error from org.apache.hadoop.hive.ql.exec.mr.MapRedTa ...

随机推荐

  1. shiro学习笔记

    一.概念: shiro是apache旗下一个开源框架,它将软件系统的安全认证相关的功能抽取出来,实现用户身份认证.权限授权.加密.会话管理等功能,组成了一个通用的安全认证框架. (一)shiro的功能 ...

  2. react 学习资料

    react 学习资料 项目 学习资料 react 中文版:https://doc.react-china.org/ react-router https://reacttraining.com/rea ...

  3. pyquery的使用

    常用的三种初始化方法: 1.字符串初始化: from pyquery import PyQuery as pq html=""" <html> <hea ...

  4. 配置hive元数据数据库

    <configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <v ...

  5. Nginx + Tomcat搭建集群

    一.Tomcat集群带来的好处 1.提高服务的性能,并发能力,以及高可用性 2.提供项目架构的横向扩展能力 二.Tomcat集群实现原理 通过Nginx负载均衡进行请求转发 三.Nginx + Tom ...

  6. dragino2 ar9331将LED管脚当做普通gpio使用

    1. 将gpio13和gpio17也当作普通gpio使用,修改 target/linux/ar71xx/files/arch/mips/ath79/mach-dragino2.c文件如下 /* * D ...

  7. HD-SDI制式学习

    HD-SDI制式学习 1.资料来源(XAPP514) 2.HD-SDI相关知识 需要注意的是每一行的开始为EAV,序列依次为:EAV Blank Video 3.XYZ的格式 4.Line numbe ...

  8. Avalon总线学习 ---Avalon Interface Specifications

    Avalon总线学习 ---Avalon Interface Specifications 1.Avalon Interfaces in a System and Nios II Processor ...

  9. nginx-ngx_http_random_index_module

    模块简介:在收到以"/"为结尾的请求时,在对应的目录下随机选择一个文件作为index文件. 模块类型:常用标准http模块 使用实例: location /privacy_poli ...

  10. BlockingQueue队列

    1.BlockingQueue定义的常用方法如下     抛出异常 特殊值 阻塞 超时 插入 add(e) offer(e) put(e) offer(e,time,unit) 移除 remove() ...