问题:

将MongoDB数据导入Hive,按照https://blog.csdn.net/thriving_fcl/article/details/51471248文章,在hive建外部表与mongodb做映射后,执行后出现

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON

建表语句如下:

CREATE EXTERNAL TABLE mongotohive
  id string,
  userid string,
  age bigint,
  status string
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","userid":"user_id","age":"age","status":"status"}')
TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/mydb.users'); 

mongodb 数据如下:

db.users.find()
{ "_id" : ObjectId("5b456e33a93daf7ae53e6419"), "user_id" : "abc123", "age" : 58, "status" : "D" }
{ "_id" : ObjectId("5b45705ca93daf7ae53e8b2a"), "user_id" : "bcd001", "age" : 45, "status" : "C" }

解决方案:

将mongo-hadoop-core-2.0.0.jar、mongo-hadoop-hive-2.0.0.jar、mongo-java-driver-3.7.1.jar三个jar包放到hive的lib文件夹下后,再次运行成功。如下:

hive> CREATE EXTERNAL TABLE mongotohive
    > ( 
    >   id string,
    >   userid string,
    >   age bigint,
    >   status string
    > )
    > STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
    > WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","userid":"user_id","age":"age","status":"status"}')
    > TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/mydb.users');
OK
Time taken: 1.431 seconds
hive> select * from mongotohive;
OK
5b456e33a93daf7ae53e6419        abc123  58      D
5b45705ca93daf7ae53e8b2a        bcd001  45      C
Time taken: 0.601 seconds, Fetched: 2 row(s)
hive> 

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON的更多相关文章

  1. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: An exception was thrown while adding/validating class(es) :

    在hive命令行创建表时报错: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. ...

  2. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(me

    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(me ...

  3. Hive的Shell里hive> 执行操作时,出现FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask错误的解决办法(图文详解)

    不多说,直接上干货! 这个问题,得非 你的hive和hbase是不是同样都是CDH版本,还是一个是apache版本,一个是CDH版本. 问题详情 [kfk@bigdata-pro01 apache-h ...

  4. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

    hive启动后,出现以下异常 hive> show databases; FAILED: Error / failed on connection exception: java.net.Con ...

  5. hive_异常_01_(未解决)FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V

    一.如果出现如下错误需要编译源码 需要重新编译Hbase-handler源码 步骤如下: 准备Jar包: 将Hbase 中lib下的jar包和Hive中lib下的jar包全部导入到一起. 记得删除里面 ...

  6. hive遇到FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误

    hive遇到FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误 起因 ...

  7. Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/

    Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/ ...

  8. 解决hiveserver2报错:java.io.IOException: Job status not available - Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

    用户使用的sql: select count( distinct patient_id ) from argus.table_aa000612_641cd8ce_ceff_4ea0_9b27_0a3a ...

  9. java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

    执行Hive查询: Console是这样报错的 java.sql.SQLException: Error from org.apache.hadoop.hive.ql.exec.mr.MapRedTa ...

随机推荐

  1. ESB雏形 -- 项目企业服务总线初始

    今天要厚着脸皮给大家推荐一个自己做的通信中间件——ServiceAnt,目前已经在我们团队的两个产品线上投入了使用. ServiceAnt是什么 它最初的定位是ESB(企业服务总线),但目前还没有达到 ...

  2. 通过 DDNS 解决宽带拨号 ip 变化问题

    前面你的文章我已经写了 写了 DMZ 内网映射的 方式. 这样内网主机已经暴露在外网中了. 但是 拨号上网我们的ip是 会变化的.大概规律就是 每次拨号都会变化.如果不拨号,每 24 小时 ip也会自 ...

  3. yum和编译两种方式升级or降级Centos内核

    http://blog.51cto.com/renzhiyuan/1882599 今天探讨用yum和编译两种方式升级或者降级内核版本: 升级:比如玩kvm,docker等虚拟化,centos内核则升级 ...

  4. Jenkins进阶-Slave 节点的配置(11)

    Jenkins的主节点无法满足当前的发布需求,所以我们需要增加一个从节点,我们是本着节约的模式,所以主从节点均需要有任务来进行构建: 1.系统管理->管理节点: 2.添加从节点,管理节点-> ...

  5. Android Monkey测试入门

    第一步:搭建环境:主要是安装和搭建java和sdk环境,说白了,对我们安卓开发来说,只要搭建好了Android开发环境,Monkey测试环境基本就是OK的了.可以参考:http://www.cnblo ...

  6. py-day1-6 python 5个灰魔法 【len,index索引,for循环,切片】

    # 索引,下标,获取字符串中的某一个字符. test = 'MuMingJun' v = test[3] print(v) i # 切片 test = 'MuMingJun' v = test[0:- ...

  7. 路由和HTTP方法

    from flask import Flask app = Flask(__name__) # # 路由用法 # @app.route('/') def hello_world(): # 这种方法再不 ...

  8. xml.sax 笔记

    from xml.sax import saxutils html_str = """<!DOCTYPE html> <html> <hea ...

  9. HanLP 关键词提取算法分析详解

    HanLP 关键词提取算法分析详解 l 参考论文:<TextRank: Bringing Order into Texts> l TextRank算法提取关键词的Java实现 l Text ...

  10. ML平台_设计要点

    如果说机器是人类手的延伸.交通工具是人类腿的延伸,那么人工智能就是人类大脑的延伸,甚至可以帮助人类自我进化,超越自我.人工智能也是计算机领域最前沿和最具神秘色彩的学科,科学家希望制造出代替人类思考的智 ...