hive的窗口函数ntile、row_number、rank

一、ntile

序列函数不支持window子句

数据准备：

cookie1,--,
cookie1,--,
cookie1,--,
cookie1,--,
cookie1,--,
cookie1,--,
cookie1,--,
cookie2,--,
cookie2,--,
cookie2,--,
cookie2,--,
cookie2,--,
cookie2,--,
cookie2,--,

把数据load到表中

hive (default)> select * from cookie;
OK
cookie.cookieid    cookie.createtime    cookie.pv
cookie1    --
cookie1    --
cookie1    --
cookie1    --
cookie1    --
cookie1    --
cookie1    --
cookie2    --
cookie2    --
cookie2    --
cookie2    --
cookie2    --
cookie2    --
cookie2    --
Time taken: 0.086 seconds, Fetched:  row(s)

NTILE(n)，用于将分组数据按照顺序切分成n片，返回当前切片值
NTILE不支持ROWS BETWEEN，比如 NTILE(2) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN 3 PRECEDING AND CURRENT ROW)
如果切片不均匀，默认增加第一个切片的分布

select cookieid,createtime,pv,
ntile(2) over(partition by cookieid order by createtime) as rn1,--分组内将数据分为2片
ntile(3) over(partition by cookieid order by createtime) as rn2,--分组内将数据分为3片
ntile(4) over(order by createtime) as rn3
from cookie order by cookieid,createtime;

结果是：

OK
cookieid    createtime    pv    rn1    rn2    rn3
cookie1        2015-04-10    1    1    1    1
cookie1        2015-04-11    5    1    1    1
cookie1        2015-04-12    7    1    1    2
cookie1        2015-04-13    3    1    2    2
cookie1        2015-04-14    2    2    2    3
cookie1        2015-04-15    4    2    3    4
cookie1        2015-04-16    4    2    3    4
cookie2        2015-04-10    2    1    1    1
cookie2        2015-04-11    3    1    1    1
cookie2        2015-04-12    5    1    1    2
cookie2        2015-04-13    6    1    2    2
cookie2        2015-04-14    3    2    2    3
cookie2        2015-04-15    9    2    3    3
cookie2        2015-04-16    7    2    3    4
Time taken: 76.302 seconds, Fetched: 14 row(s)

比如，统计一个cookie，pv数最多的前1/3的天

select cookieid,createtime,pv,
ntile(3) over(partition by cookieid order by pv desc) as rn
from cookie;

结果是：

OK
cookieid    createtime    pv    rn
cookie1        2015-04-12    7    1
cookie1        2015-04-11    5    1
cookie1        2015-04-16    4    1
cookie1        2015-04-15    4    2
cookie1        2015-04-13    3    2
cookie1        2015-04-14    2    3
cookie1        2015-04-10    1    3
cookie2        2015-04-15    9    1
cookie2        2015-04-16    7    1
cookie2        2015-04-13    6    1
cookie2        2015-04-12    5    2
cookie2        2015-04-11    3    2
cookie2        2015-04-14    3    3
cookie2        2015-04-10    2    3
Time taken: 22.654 seconds, Fetched: 14 row(s)

rn=1就是我们想要的结果

 二、ROW_NUMBER

ROW_NUMBER()–从1开始，按照顺序，生成分组内记录的序列
–比如，按照pv降序排列，生成分组内每天的pv名次
ROW_NUMBER() 的应用场景非常多，再比如，获取分组内排序第一的记录;获取一个session中的第一条refer等。

select cookieid,createtime,pv,
row_number() over(partition by cookieid order by pv desc) as rn
from cookie;

结果：

OK
cookieid    createtime    pv    rn
cookie1    2015-04-12    7    1
cookie1    2015-04-11    5    2
cookie1    2015-04-16    4    3
cookie1    2015-04-15    4    4
cookie1    2015-04-13    3    5
cookie1    2015-04-14    2    6
cookie1    2015-04-10    1    7
cookie2    2015-04-15    9    1
cookie2    2015-04-16    7    2
cookie2    2015-04-13    6    3
cookie2    2015-04-12    5    4
cookie2    2015-04-11    3    5
cookie2    2015-04-14    3    6
cookie2    2015-04-10    2    7
Time taken: 22.657 seconds, Fetched: 14 row(s)

三、rank和dense_rank

—RANK() 生成数据项在分组中的排名，排名相等会在名次中留下空位
—DENSE_RANK() 生成数据项在分组中的排名，排名相等会在名次中不会留下空位

select cookieid,createtime,pv,
rank() over(partition by cookieid order by pv desc) as rn1,
dense_rank() over(partition by cookieid order by pv desc) as rn2,
row_number() over(partition by cookieid order by pv desc) as rn3
from cookie
where cookieid='cookie1';

结果：

OK
cookieid    createtime    pv    rn1    rn2    rn3
cookie1    2015-04-12    7    1    1    1
cookie1    2015-04-11    5    2    2    2
cookie1    2015-04-16    4    3    3    3
cookie1    2015-04-15    4    3    3    4
cookie1    2015-04-13    3    5    4    5
cookie1    2015-04-14    2    6    5    6
cookie1    2015-04-10    1    7    6    7
Time taken: 23.841 seconds, Fetched: 7 row(s)

rn1: 15号和16号并列第3, 13号排第5
rn2: 15号和16号并列第3, 13号排第4
rn3: 如果相等，则按记录值排序，生成唯一的次序，如果所有记录值都相等，或许会随机排吧。