Hive创建HBase，ES外部表

1、创建HBase外部表

CREATE EXTERNAL TABLE `ods_women`(
  `rowkey` string COMMENT 'from deserializer',
  `article` string COMMENT 'from deserializer',
  `url` string COMMENT 'from deserializer',
  `web` string COMMENT 'from deserializer',
  `keyword` string COMMENT 'from deserializer',
  `acquire_time` string COMMENT 'from deserializer',
  `article_time` string COMMENT 'from deserializer',
  `calculate_time` string COMMENT 'from deserializer',
  `title` string COMMENT 'from deserializer',
  `english_industry` string COMMENT 'from deserializer')
STORED BY
  'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
'hbase.columns.mapping'=':key,info:article_word,info:article_url,info:website,info:chinese_keyword,info:acquire_time,info:article_time,info:calculate_time,info:article_title,info:english_industry')
TBLPROPERTIES ( 'hbase.table.name'='test:ods_women');

2、创建es外部表

1）下载相关jar包

https://www.elastic.co/cn/downloads/past-releases#es-hadoop

2）hive命令行输入

add jar /home/jar/elasticsearch-hadoop-5.5.3/dist/elasticsearch-hadoop-5.5.3.jar;

3）创建表

drop table if exists dw_women_article_core;
create external table dw_women_article_core(
md5id string,
article_id string,
keyword string,
search_keyword string,
keyword_weight bigint,
article_title string,
article_content string,
web string,
article_date string,
status bigint,
keyword_push string
)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.nodes' = '192.168.2.14:9200',
'es.index.auto.create' = 'true',
'es.resource' = 'app_knowledgegraph_new/app_women_article_core',
'es.mapping.id' = 'md5id',
'es.mapping.names' = 'md5id:md5id,article_id:article_id,keyword:keyword,search_keyword:search_keyword,keyword_weight:keyword_weight,article_title:article_title,web:web,article_date:article_date,status:status,keyword_push:keyword_push,article_content:article_content',
'es.nodes.wan.only' = 'true');

es.index.auto.create (default yes)

Whether elasticsearch-hadoop should create an index (if its missing) when writing data to Elasticsearch or fail.

es.nodes.wan.only (default false)

Whether the connector is used against an Elasticsearch instance in a cloud/restricted environment over the WAN, such as Amazon Web Services. In this mode, the connector disables discovery and only connects through the declared es.nodes during all operations, including reads and writes. Note that in this mode, performance is highly affected.