hive splict, explode, lateral view, concat_ws
hive> create table arrays (x array<string>)
> row format delimited fields terminated by '\001'
> collection items terminated by '\002'
> ;
OK
Time taken: 0.574 seconds
hive> show tables;
OK
arrays
jigou
Time taken: 0.15 seconds, Fetched: 2 row(s)
hive> show create table arrays;
OK
CREATE TABLE `arrays`(
`x` array<string>)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\u0001'
COLLECTION ITEMS TERMINATED BY '\u0002'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
'hdfs://hdp1.hadoop.test:8020/apps/hive/warehouse/zhangchao.db/arrays'
TBLPROPERTIES (
'transient_lastDdlTime'='1441481876')
Time taken: 0.399 seconds, Fetched: 13 row(s)
hive> load data local inpath './arrays.text' into arrays;
FAILED: ParseException line 1:44 missing TABLE at 'arrays' near '<EOF>'
hive> load data local inpath './arrays.text' into table arrays;
FAILED: SemanticException Line 1:23 Invalid path ''./arrays.text'': No files matching path file:/usr/java/arrays.text
hive> load data local inpath '/home/zhangchao/arrays.text' into table arrays;
Loading data to table zhangchao.arrays
Table zhangchao.arrays stats: [numFiles=1, totalSize=10]
OK
Time taken: 1.322 seconds
hive> select * from arrays;
OK
["a","b"]
["c","d","e"]
Time taken: 1.076 seconds, Fetched: 2 row(s)
hive> select explode(x) as xx from arrays;
Query ID = zhangchao_20150906052727_87971c63-6a8a-4a15-9465-2564cf22c837
Total jobs = 1
Launching Job 1 out of 1 Status: Running (Executing on YARN cluster with App id application_1440440163499_0008) --------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 1 1 0 0 3 0
--------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 11.95 s
--------------------------------------------------------------------------------
OK
a
b
c
d
e
Time taken: 19.465 seconds, Fetched: 5 row(s)
hive> select concat_ws(',', '1','2','3','4') from arrays;
OK
1,2,3,4
1,2,3,4
Time taken: 0.107 seconds, Fetched: 2 row(s)
hive> select split(concat_ws(',', '1','2','3','4'),',') from arrays;
OK
["1","2","3","4"]
["1","2","3","4"]
Time taken: 0.128 seconds, Fetched: 2 row(s)
hive> select split(concat_ws(',', '1','2','3','4'),',')[3] from arrays;
OK
4
4
Time taken: 0.116 seconds, Fetched: 2 row(s)
hive> select explode(split(concat_ws(',', '1','2','3','4'),',')) from arrays;
Query ID = zhangchao_20150906063535_e0067b77-0481-48aa-b3dc-055a7e6b6c3c
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1440440163499_0012)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 1 1 0 0 1 0
--------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 10.51 s
--------------------------------------------------------------------------------
OK
1
2
3
4
1
2
3
4
Time taken: 11.757 seconds, Fetched: 8 row(s)
hive> select * ,sp from arrays lateral view explode(split(concat_ws(',','1','2','3','4'),',')) a as sp;
Query ID = zhangchao_20150906064040_c275e654-c7d1-45d2-86a8-18fb5506d4f2
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1440440163499_0012)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 1 1 0 0 1 0
--------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 5.94 s
--------------------------------------------------------------------------------
OK
["a","b"] 1 1
["a","b"] 2 2
["a","b"] 3 3
["a","b"] 4 4
["c","d","e"] 1 1
["c","d","e"] 2 2
["c","d","e"] 3 3
["c","d","e"] 4 4
Time taken: 6.906 seconds, Fetched: 8 row(s)
hive> select 'xx' ,sp from arrays lateral view explode(split(concat_ws(',','1','2','3','4'),',')) a as sp;
Query ID = zhangchao_20150906064545_f1ce3669-80ff-45a5-a1a8-6211f56d77bd
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1440440163499_0012)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 1 1 0 0 2 0
--------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 7.20 s
--------------------------------------------------------------------------------
OK
xx 1
xx 2
xx 3
xx 4
xx 1
xx 2
xx 3
xx 4
Time taken: 8.23 seconds, Fetched: 8 row(s)
hive> select * from lateral_test;
OK
999
Time taken: 0.087 seconds, Fetched: 1 row(s)
hive> select * , sp from lateral_test lateral view explode(split(concat_ws(',','1','2','3','4','5'),',')) as sp;
FAILED: ParseException line 1:104 extraneous input 'sp' expecting EOF near '<EOF>'
hive> select * , sp from lateral_test lateral view explode(split(concat_ws(',','1','2','3','4','5'),',')) a as sp;
Query ID = zhangchao_20150906065252_ed187ad7-b400-4b71-add9-c7dc005f4af1
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established. Status: Running (Executing on YARN cluster with App id application_1440440163499_0014) --------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 1 1 0 0 1 0
--------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 7.21 s
--------------------------------------------------------------------------------
OK
999 1 1
999 2 2
999 3 3
999 4 4
999 5 5
Time taken: 16.504 seconds, Fetched: 5 row(s)
hive> select lateral_test.* , sp from lateral_test lateral view explode(split(concat_ws(',','1','2','3','4','5'),',')) a as sp;
Query ID = zhangchao_20150906065757_2db20ede-33d6-467f-886c-574a06995041
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1440440163499_0014)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 7.01 s
--------------------------------------------------------------------------------
OK
999 1
999 2
999 3
999 4
999 5
Time taken: 8.257 seconds, Fetched: 5 row(s)
hive splict, explode, lateral view, concat_ws的更多相关文章
- hive中的lateral view 与 explode函数的使用
hive中的lateral view 与 explode函数的使用 背景介绍: explode与lateral view在关系型数据库中本身是不该出现的. 因为他的出现本身就是在操作不满足第一范式的数 ...
- hive中，lateral view 与 explode函数
hive中常规处理json数据,array类型json用get_json_object(#,"$.#")这个方法足够了,map类型复合型json就需要通过数据处理才能解析. exp ...
- Hive之侧视图(Lateral View)
Lateral View和UDTF类功能函数一起使用,表中的每一行和UDTF函数输出的每一行进行连接,生成一张新的虚拟表,可以对UDTF产生的记录设置字段名称,新加的字段可以使用在sort by,gr ...
- hive中的 lateral view
lateral view用于和split, explode等UDTF一起使用,它能够将一列数据拆成多行数据,在此基础上可以对拆分后的数据进行聚合. 一个简单的例子,假设我们有一张表pageAds,它有 ...
- hive lateral view 与 explode详解
ref:https://blog.csdn.net/bitcarmanlee/article/details/51926530 1.explode hive wiki对于expolde的解释如下: e ...
- lateral view
原文地址:https://cwiki.apache.org/confluence/display/Hive/LanguageManual+LateralView# lateral view用于和spl ...
- hive之案例分析(grouping sets，lateral view explode, concat_ws)
有这样一组搜索结果数据: 租户,平台, 登录用户, 搜索关键词, 搜索的商品结果List {"tenantcode":"", "platform&qu ...
- Hive之explode和lateral view
Hive之explode 一. explode, 行转列. 1.1. 用于array类型的数据 table_name 表名 array_col 为数组类型的字段 new_col array_col被e ...
- Hive lateral view explode
select 'hello', x from dual lateral view explode(array(1,2,3,4,5)) vt as x 结果是: hello 1 hello 2 ...
随机推荐
- Android StageFrightMediaScanner源码解析
1. 简单介绍 Android中在StageFrightMediaScanner实现对多媒体文件的处理. 此外在StageFrightMediaScanner定义了支持的多媒体文件类型. 文件位置 f ...
- ES6学习笔记一：let、const、解构赋值
转载请注明原文地址:http://www.cnblogs.com/ygj0930/p/7236342.html 感谢阮一峰老师的无私奉献,开源推动世界!教程地址:http://es6.ruanyife ...
- 《Cocos2d-JS开发之旅》重印在即，感谢大家的支持
3月第一次印刷的<Cocos2d-JS开发之旅>已经销售完毕,即将启动第二次印刷. 感谢各位读者的支持,最近<开发之旅>荣登京东cocos2d-x系列书籍的销售排行首位. 新版 ...
- Oracle Data Integrator 12c----包(Package)
1 创建"包" Designer->项目->ODI_Exercise ->第一个文件夹->包,右键"新建程序包": "定义&q ...
- 一起talk C栗子吧（第八十五回：C语言实例--使用信号进行进程间通信二）
各位看官们,大家好,上一回中咱们说的是使用信号进行进程间通信的样例,这一回咱们接着上一回的内容,继续说该样例.闲话休提.言归正转. 让我们一起talk C栗子吧. 我们在上一回中举了使用信号进行进程间 ...
- ssh无法登录,提示Pseudo-terminal will not be allocated because stdin is not a terminal.
当远程通过ssh登录主机时出现Pseudo-terminal will not be allocated because stdin is not a terminal. 错误 字面意思是伪终端将 ...
- 一个进程(Process)最多可以生成多少个线程(Thread)
1.进程中创建线程的限制 默认情况下,一个线程的栈要预留1M的内存空间,而一个进程中可用的内存空间只有2G,所以理论上一个进程中最多可以开2048个线程,但是内存当然不可能完全拿来作线程的栈,所以实际 ...
- JavaScript 风格指南
来源于: https://github.com/alivebao/clean-code-js 目录 介绍 变量 函数 对象和数据结构 类 测试 并发 错误处理 格式化 注释 介绍 作者根据 Rober ...
- 你想要的iOS 小技巧总结
UITableView的Group样式下顶部空白处理 //分组列表头部空白处理 UIView *view = [[UIView alloc] initWithFrame:CGRectMake(, , ...
- Jetty使用内存过大的解决方案
之前用Jetty做过一个消息通知服务器,主要功能就是其他各个子系统如果有需要push给客户端消息的就把这个消息发给我的Server,我用WebSocket来推送给客户端~ 程序上线一段时间之后运维工程 ...