Apache Flink: Powered by Flink https://flink.apache.org/poweredby.html

Powered by Flink

Apache Flink powers business-critical applications in many companies and enterprises around the globe. On this page, we present a few notable Flink users that run interesting use cases in production and link to resources that discuss their applications in more detail.

More Flink users are listed in the Powered by Flink directory in the project wiki. Please note that the list is not comprehensive. We only add users that explicitly ask to be listed.

If you would you like to be included on this page, please reach out to the Flink user mailing list and let us know.

Alibaba, the world's largest retailer, uses a fork of Flink called Blink to optimize search rankings in real time.

Read more on the Zalando Tech Blog

访客日志分析

[root@a ~]# ll -ash /data/visitlog/
total 3.2G
20K drwxr-xr-x 2 nginx nginx 20K Oct 16 00:00 .
4.0K drwxr-xr-x 12 root root 4.0K Oct 12 09:41 ..
48M -rw-r--r-- 1 nginx nginx 48M Sep 18 23:59 visit-2018-09-18
66M -rw-r--r-- 1 nginx nginx 66M Sep 19 23:59 visit-2018-09-19
64M -rw-r--r-- 1 nginx nginx 64M Sep 20 23:59 visit-2018-09-20
68M -rw-r--r-- 1 nginx nginx 68M Sep 21 23:59 visit-2018-09-21
70M -rw-r--r-- 1 nginx nginx 70M Sep 22 23:59 visit-2018-09-22
63M -rw-r--r-- 1 nginx nginx 63M Sep 23 23:59 visit-2018-09-23
62M -rw-r--r-- 1 nginx nginx 62M Sep 24 23:59 visit-2018-09-24
70M -rw-r--r-- 1 nginx nginx 70M Sep 25 23:59 visit-2018-09-25
74M -rw-r--r-- 1 nginx nginx 74M Sep 26 23:59 visit-2018-09-26
78M -rw-r--r-- 1 nginx nginx 78M Sep 27 23:59 visit-2018-09-27
102M -rw-r--r-- 1 nginx nginx 102M Sep 28 23:59 visit-2018-09-28
125M -rw-r--r-- 1 nginx nginx 125M Sep 29 23:59 visit-2018-09-29
123M -rw-r--r-- 1 nginx nginx 123M Sep 30 23:59 visit-2018-09-30
114M -rw-r--r-- 1 nginx nginx 114M Oct 1 23:59 visit-2018-10-01
112M -rw-r--r-- 1 nginx nginx 112M Oct 2 23:59 visit-2018-10-02
120M -rw-r--r-- 1 nginx nginx 120M Oct 3 23:59 visit-2018-10-03
119M -rw-r--r-- 1 nginx nginx 119M Oct 4 23:59 visit-2018-10-04
128M -rw-r--r-- 1 nginx nginx 128M Oct 5 23:59 visit-2018-10-05
126M -rw-r--r-- 1 nginx nginx 126M Oct 6 23:59 visit-2018-10-06
124M -rw-r--r-- 1 nginx nginx 124M Oct 7 23:59 visit-2018-10-07
141M -rw-r--r-- 1 nginx nginx 141M Oct 8 23:59 visit-2018-10-08
141M -rw-r--r-- 1 nginx nginx 141M Oct 9 23:59 visit-2018-10-09
146M -rw-r--r-- 1 nginx nginx 146M Oct 10 23:59 visit-2018-10-10
137M -rw-r--r-- 1 nginx nginx 137M Oct 11 23:59 visit-2018-10-11
144M -rw-r--r-- 1 nginx nginx 144M Oct 12 23:59 visit-2018-10-12
139M -rw-r--r-- 1 nginx nginx 139M Oct 13 23:59 visit-2018-10-13
196M -rw-r--r-- 1 nginx nginx 196M Oct 14 23:59 visit-2018-10-14
245M -rw-r--r-- 1 nginx nginx 245M Oct 15 23:59 visit-2018-10-15
119M -rw-r--r-- 1 nginx nginx 119M Oct 16 11:40 visit-2018-10-16
[root@a ~]#

[root@a ~]# head /data/visitlog/visit-2018-10-15

{"uid":50007901,"uuid":"61408c43-b4f5-4614-86a8-aebbc3943820","ip":"3071402833","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/ZxJClxN76LzgYWJD-vQAIA.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":6,"city":77,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50017122,"uuid":"4fe41ece-e28a-4ba1-871e-8dfecbfc1d25","ip":"2004914835","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/S5vPXX4IK3PJQzipD_JbRg.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":6,"city":79,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50012077,"uuid":"28d1c40b-3d01-47bb-9532-abd39b2e1600","ip":"608159022","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/u2JljIAn5juvLng5oaIx8Q.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":3,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":34056498,"uuid":"bf0d9674-8835-497d-88eb-75b1a12a88fb","ip":"1971867257","site":70,"source":0,"address":"http:\/\/www.jqw.com\/bianmin\/m\/33239101.html","engine":2,"referer":"https:\/\/m.baidu.com\/from=1014517c\/bd_page_type=1\/ssid=0\/uid=0\/pu=sz%401320_480%2Ccuid%40_u20tguaHugMu-iqYa-3igPq-ilAaSi10827ag8m28KaLqqqB%2Ccua%40_avLC_aE2f4qywoUfpw1z4aS2N_VaXiHoavRhGQHNqqqB%2Ccut%40lfSpI0tI2iyjaXiDyuvGC5kcSMzWtqqSB%2Cosname%40baiduboxapp%2Cctv%402%2Ccfrom%401014517c%2Ccen%40cuid_cua_cut%2Ccsrc%40bdbox_tserch_txt%2Cta%40zbios_2_6.0_6_10.10%2Cusm%402%2Cvmgdb%400020100228y\/baiduid=1CC91FA18C1983E5C17C8EEF12E0C62E\/w=0_10_\/t=zbios\/l=1\/tc?ref=www_zbios&lid=8867811778563143063&order=2&fm=alop&waplogo=1&is_baidu=0&h5ad=0&tj=www_normal_2_0_10_title&vit=osres&waput=3&cltj=normal_title&asres=1&title=%E9%9D%92%E5%B2%9B%E5%A4%96%E5%A2%99%E4%BF%9D%E6%B8%A9%E6%96%BD%E5%B7%A5%2C%E9%98%B2%E6%B0%B4%E6%96%BD%E5%B7%A5%E4%B8%93%E4%B8%9A%E4%BB%8E%E4%BA%8B%E5%BB%BA%E7%AD%91%E5%8A%A0%E5%9B%BA-%E9%87%91%E6%B3%89%E7%BD%91&hwj=1603686807417560&dict=-1&wd=&eqid=7b10cbd43447f400100000015bc367c5&w_qd=IlPT2AEptyoA_ykwsPIcyRexDE6OlY9oqjwYlBPVtg5rXvUzNE-dyJzfkxJ7m_oU5xJuDcK&tcplug=1&sec=33447&di=7de4c89b4ec3ed81&bdenc=1&nsrc=IlPT2AEptyoA_yixCFOxXnANedT62v3IEQGG_yRJBXSxokDyqRLhGNVoXzb62Sm5HoCb9jHRsxw1wXSaAT-il17","keyword":"","browser":98,"language":37,"screen_color":33,"screen_size":0,"system":13,"platform":62,"operator":2,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":34021738,"uuid":"b00ae536-4ca5-474b-ba2b-fddd4c0b9674","ip":"1879278394","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/53236367.html","engine":0,"referer":"http:\/\/www.ijntv.cn\/inews\/53236367.html","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":2,"country":1,"province":16,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50017547,"uuid":"a76935f7-1da9-4dca-b95f-ba0f298ff5f5","ip":"1972898818","site":0,"source":0,"address":"http:\/\/www.dongfeng.net\/make\/6QRkYJX037Pkyw1co6-DPw.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":2,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539532801,"create_date":"2018-10-15 00:00:01"}

{"uid":50010757,"uuid":"dda000f3-930d-4e2a-8834-7edd05230574","ip":"1018017398","site":0,"source":0,"address":"http:\/\/e.hbqnb.com\/inews\/53383689.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":3,"city":45,"flash":"0","java":"0","request_time":1539532801,"create_date":"2018-10-15 00:00:01"}

{"uid":34004741,"uuid":"ef59dadf-fb58-46fa-aaf1-c1eb8a57e873","ip":"3071113907","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/55373856.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":6,"city":77,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

{"uid":50010336,"uuid":"65b0ac4f-2172-4746-a82e-fc8240240755","ip":"1996541572","site":0,"source":0,"address":"http:\/\/e.hbqnb.com\/inews\/53402956.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":0,"country":1,"province":8,"city":0,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

{"uid":34050322,"uuid":"f9dd0bda-5416-4535-b08b-d91c196977af","ip":"1962504456","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/5TJU_3pIEFfcni314q80ZA.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":30,"city":0,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

[root@a ~]# tail /data/visitlog/visit-2018-10-15

{"uid":50001890,"uuid":"d1ed8885-280c-4f5d-a06b-112ac2c78b01","ip":"1909901422","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/grFFl7D-XfwzTXUuHIyFiA.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":0,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539619195,"create_date":"2018-10-15 23:59:55"}

{"uid":50005483,"uuid":"6dcf7b2a-dcd9-460f-8213-0dbe220d474c","ip":"974469123","site":0,"source":0,"address":"http:\/\/www.xichu.net\/look\/45101554.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":3,"country":1,"province":212,"city":0,"flash":"0","java":"0","request_time":1539619196,"create_date":"2018-10-15 23:59:56"}

{"uid":50013267,"uuid":"46245679-8699-48ff-abe2-f691d3f14aa0","ip":"1943015812","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/PMekyfhQyRj8Qzu81ZD6Ww.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":31,"city":384,"flash":"0","java":"0","request_time":1539619197,"create_date":"2018-10-15 23:59:57"}

{"uid":50009806,"uuid":"75d15fb3-8f90-409c-ae16-f6dd58b83979","ip":"3733153632","site":0,"source":0,"address":"http:\/\/www.metalnews.cn\/xinxi\/xlOGEPlObiXPgZGG1ZTFxg.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":0,"country":1,"province":2,"city":0,"flash":"0","java":"0","request_time":1539619197,"create_date":"2018-10-15 23:59:57"}

{"uid":50015377,"uuid":"e70a8aaa-6798-4a49-a3ef-93ff0f6b875c","ip":"2090975143","site":0,"source":0,"address":"http:\/\/www.zjkonline.com\/well\/ycEzO-8D4LBWdiCA6kjb2w.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":3,"country":1,"province":26,"city":342,"flash":"0","java":"0","request_time":1539619198,"create_date":"2018-10-15 23:59:58"}

{"uid":50014416,"uuid":"e7441c1d-f40e-4851-9220-58ed03a416db","ip":"3060659990","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/HYSAdHafvcC5TeNfN_Q9fQ.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":17,"city":0,"flash":"0","java":"0","request_time":1539619198,"create_date":"2018-10-15 23:59:58"}

{"uid":50016615,"uuid":"32185523-813f-42b2-aca6-5fb03d0a71a0","ip":"2008640953","site":0,"source":0,"address":"http:\/\/www.zjkonline.com\/well\/QToFaPQVeQ9ix2EuIAp03Q.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":3,"country":1,"province":22,"city":293,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":34032314,"uuid":"315ea4eb-8fa1-4430-a38f-bcbeba450897","ip":"827363088","site":0,"source":0,"address":"http:\/\/www.01hr.com\/bfdx\/45995002.html","engine":0,"referer":"","keyword":"","browser":5,"language":0,"screen_color":34,"screen_size":0,"system":67,"platform":60,"operator":1,"country":1,"province":16,"city":226,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":34032314,"uuid":"315ea4eb-8fa1-4430-a38f-bcbeba450897","ip":"3026902583","site":0,"source":0,"address":"http:\/\/www.01hr.com\/bfdx\/45995002.html","engine":0,"referer":"http:\/\/www.01hr.com\/bfdx\/45995002.html","keyword":"","browser":5,"language":0,"screen_color":34,"screen_size":0,"system":67,"platform":60,"operator":1,"country":1,"province":16,"city":221,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":50008572,"uuid":"0fa00371-f97a-429f-bfdd-5bdfdb5b6f92","ip":"3747928108","site":0,"source":0,"address":"http:\/\/www.xxcmw.com\/shangxun\/55410287.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":2,"country":1,"province":18,"city":0,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

[root@a ~]#

当前场景借助负载均衡，对网站的访问分发到3个web服务器，同时各自访客日志落入相应web服务器（日志没有做集中化处理）

原方案是借助日志上报，给阿里的日志服务，然后在投入到阿里的dataworks中计算，最后写入mysql

现方案借助开源组件，实现mapReduce过程，得到汇总数据；

同时提高实时性。

Powered by Flink的更多相关文章

腾讯大数据平台Oceanus: A one-stop platform for real time stream processing powered by Apache Flink
January 25, 2019Use Cases, Apache Flink The Big Data Team at Tencent In recent years, the increa ...
Flink应用案例：How Trackunit leverages Flink to process real-time data from industrial IoT devices
January 22, 2019Use Cases, Apache Flink Lasse Nedergaard Recently there has been significant dis ...
Flink写入kafka时，只写入kafka的部分Partitioner，无法写所有的Partitioner问题
1. 写在前面在利用flink实时计算的时候,往往会从kafka读取数据写入数据到kafka,但会发现当kafka多个Partitioner时,特别在P量级数据为了kafka的性能kafka的节点有 ...
从零构建Flink SQL计算平台 - 1平台搭建
一.理想与现实 Apache Flink 是一个分布式流批一体化的开源平台.Flink 的核心是一个提供数据分发.通信以及自动容错的流计算引擎.Flink 在流计算之上构建批处理,并且原生的支持迭代计 ...
apache flink 入门
配置环境包括 JAVA_HOME jobmanager.rpc.address jobmanager.heap.mb 和 taskmanager.heap.mb taskmanager.number ...
Flink 1.1 – ResourceManager
Flink resource manager的作用如图, FlinkResourceManager /** * * <h1>Worker allocation steps</h1 ...
Apache Flink初接触
Apache Flink闻名已久,一直没有亲自尝试一把,这两天看了文档,发现在real-time streaming方面,Flink提供了更多高阶的实用函数. 用Apache Flink实现WordC ...
Flink - InstanceManager
InstanceManager用于管理JobManager申请到的taskManager和slots资源 /** * Simple manager that keeps track of which ...
Flink – window operator
参考, http://wuchong.me/blog/2016/05/25/flink-internals-window-mechanism/ http://wuchong.me/blog/201 ...

随机推荐

wx小程序的学习
传送门: # 微信小程序案例大全 https://www.cnblogs.com/icyhoo/p/6282574.html # 微信小程序开发工具 https://mp.weixin.qq.com/ ...
140726暑期培训.txt
1. 输入多组数据的时候 while(scanf("%s",s)!=EOF) while(gets(s)!=NULL) 用gets和scanf不 ...
544. Top k Largest Numbers【medium】
Given an integer array, find the top k largest numbers in it. Example Given [3,10,1000,-99,4,100] ...
CCNA2.0笔记_路由分类
直连路由:当在路由器上配置了接口的IP地址,并且接口状态为up的时候,路由表中就出现直连路由项静态路由:静态路由是由管理员手工配置的,是单向的. 默认路由:当路由器在路由表中找不到目标网络的路由条目 ...
vue 插件
开发插件插件通常会为vue添加全局功能,插件的范围没有限制--一般有下面几种: 1,添加全局方法或者属性,例:vue-coustom-element 2,添加全局资源:指令.过滤器,.过渡等,如vu ...
winform播放音乐
string sound = Application.StartupPath +@"\song\123.wav"; //Application.StartupPath:程序exe所 ...
STM32F10x_SPI（硬件接口 + 软件模拟）读写Flash（25Q16）
推荐分享一个大神的人工智能教程.零基础!通俗易懂!风趣幽默!还带黄段子!希望你也加入到人工智能的队伍中来! http://www.captainbed.net/strongerhuang Ⅰ.写在前 ...
页面中添加qq客服
html页面 <html> <head> <meta http-equiv="Content-Type" content="text/htm ...
git 工作模式
个人在学习Git工作流的过程中,从原有的 SVN 模式很难完全理解Git的协作模式,直到有一天我看到了下面的文章,好多遗留在心中的困惑迎刃而解: 我们以使用SVN的工作流来使用Git有什么不妥? Gi ...
java为啥计算时间从1970年1月1日开始
http://www.myexception.cn/program/1494616.html ————————————————————————————————————————————————————— ...

Powered by Flink

Powered by Flink

Powered by Flink的更多相关文章

随机推荐

热门专题