Apache Flink: Powered by Flink https://flink.apache.org/poweredby.html

Powered by Flink

Apache Flink powers business-critical applications in many companies and enterprises around the globe. On this page, we present a few notable Flink users that run interesting use cases in production and link to resources that discuss their applications in more detail.

More Flink users are listed in the Powered by Flink directory in the project wiki. Please note that the list is not comprehensive. We only add users that explicitly ask to be listed.

If you would you like to be included on this page, please reach out to the Flink user mailing list and let us know.

Alibaba, the world's largest retailer, uses a fork of Flink called Blink to optimize search rankings in real time.

Read more on the Zalando Tech Blog

访客日志分析

[root@a ~]# ll -ash /data/visitlog/
total 3.2G
20K drwxr-xr-x 2 nginx nginx 20K Oct 16 00:00 .
4.0K drwxr-xr-x 12 root root 4.0K Oct 12 09:41 ..
48M -rw-r--r-- 1 nginx nginx 48M Sep 18 23:59 visit-2018-09-18
66M -rw-r--r-- 1 nginx nginx 66M Sep 19 23:59 visit-2018-09-19
64M -rw-r--r-- 1 nginx nginx 64M Sep 20 23:59 visit-2018-09-20
68M -rw-r--r-- 1 nginx nginx 68M Sep 21 23:59 visit-2018-09-21
70M -rw-r--r-- 1 nginx nginx 70M Sep 22 23:59 visit-2018-09-22
63M -rw-r--r-- 1 nginx nginx 63M Sep 23 23:59 visit-2018-09-23
62M -rw-r--r-- 1 nginx nginx 62M Sep 24 23:59 visit-2018-09-24
70M -rw-r--r-- 1 nginx nginx 70M Sep 25 23:59 visit-2018-09-25
74M -rw-r--r-- 1 nginx nginx 74M Sep 26 23:59 visit-2018-09-26
78M -rw-r--r-- 1 nginx nginx 78M Sep 27 23:59 visit-2018-09-27
102M -rw-r--r-- 1 nginx nginx 102M Sep 28 23:59 visit-2018-09-28
125M -rw-r--r-- 1 nginx nginx 125M Sep 29 23:59 visit-2018-09-29
123M -rw-r--r-- 1 nginx nginx 123M Sep 30 23:59 visit-2018-09-30
114M -rw-r--r-- 1 nginx nginx 114M Oct 1 23:59 visit-2018-10-01
112M -rw-r--r-- 1 nginx nginx 112M Oct 2 23:59 visit-2018-10-02
120M -rw-r--r-- 1 nginx nginx 120M Oct 3 23:59 visit-2018-10-03
119M -rw-r--r-- 1 nginx nginx 119M Oct 4 23:59 visit-2018-10-04
128M -rw-r--r-- 1 nginx nginx 128M Oct 5 23:59 visit-2018-10-05
126M -rw-r--r-- 1 nginx nginx 126M Oct 6 23:59 visit-2018-10-06
124M -rw-r--r-- 1 nginx nginx 124M Oct 7 23:59 visit-2018-10-07
141M -rw-r--r-- 1 nginx nginx 141M Oct 8 23:59 visit-2018-10-08
141M -rw-r--r-- 1 nginx nginx 141M Oct 9 23:59 visit-2018-10-09
146M -rw-r--r-- 1 nginx nginx 146M Oct 10 23:59 visit-2018-10-10
137M -rw-r--r-- 1 nginx nginx 137M Oct 11 23:59 visit-2018-10-11
144M -rw-r--r-- 1 nginx nginx 144M Oct 12 23:59 visit-2018-10-12
139M -rw-r--r-- 1 nginx nginx 139M Oct 13 23:59 visit-2018-10-13
196M -rw-r--r-- 1 nginx nginx 196M Oct 14 23:59 visit-2018-10-14
245M -rw-r--r-- 1 nginx nginx 245M Oct 15 23:59 visit-2018-10-15
119M -rw-r--r-- 1 nginx nginx 119M Oct 16 11:40 visit-2018-10-16
[root@a ~]#

[root@a ~]# head /data/visitlog/visit-2018-10-15

{"uid":50007901,"uuid":"61408c43-b4f5-4614-86a8-aebbc3943820","ip":"3071402833","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/ZxJClxN76LzgYWJD-vQAIA.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":6,"city":77,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50017122,"uuid":"4fe41ece-e28a-4ba1-871e-8dfecbfc1d25","ip":"2004914835","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/S5vPXX4IK3PJQzipD_JbRg.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":6,"city":79,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50012077,"uuid":"28d1c40b-3d01-47bb-9532-abd39b2e1600","ip":"608159022","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/u2JljIAn5juvLng5oaIx8Q.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":3,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":34056498,"uuid":"bf0d9674-8835-497d-88eb-75b1a12a88fb","ip":"1971867257","site":70,"source":0,"address":"http:\/\/www.jqw.com\/bianmin\/m\/33239101.html","engine":2,"referer":"https:\/\/m.baidu.com\/from=1014517c\/bd_page_type=1\/ssid=0\/uid=0\/pu=sz%401320_480%2Ccuid%40_u20tguaHugMu-iqYa-3igPq-ilAaSi10827ag8m28KaLqqqB%2Ccua%40_avLC_aE2f4qywoUfpw1z4aS2N_VaXiHoavRhGQHNqqqB%2Ccut%40lfSpI0tI2iyjaXiDyuvGC5kcSMzWtqqSB%2Cosname%40baiduboxapp%2Cctv%402%2Ccfrom%401014517c%2Ccen%40cuid_cua_cut%2Ccsrc%40bdbox_tserch_txt%2Cta%40zbios_2_6.0_6_10.10%2Cusm%402%2Cvmgdb%400020100228y\/baiduid=1CC91FA18C1983E5C17C8EEF12E0C62E\/w=0_10_\/t=zbios\/l=1\/tc?ref=www_zbios&lid=8867811778563143063&order=2&fm=alop&waplogo=1&is_baidu=0&h5ad=0&tj=www_normal_2_0_10_title&vit=osres&waput=3&cltj=normal_title&asres=1&title=%E9%9D%92%E5%B2%9B%E5%A4%96%E5%A2%99%E4%BF%9D%E6%B8%A9%E6%96%BD%E5%B7%A5%2C%E9%98%B2%E6%B0%B4%E6%96%BD%E5%B7%A5%E4%B8%93%E4%B8%9A%E4%BB%8E%E4%BA%8B%E5%BB%BA%E7%AD%91%E5%8A%A0%E5%9B%BA-%E9%87%91%E6%B3%89%E7%BD%91&hwj=1603686807417560&dict=-1&wd=&eqid=7b10cbd43447f400100000015bc367c5&w_qd=IlPT2AEptyoA_ykwsPIcyRexDE6OlY9oqjwYlBPVtg5rXvUzNE-dyJzfkxJ7m_oU5xJuDcK&tcplug=1&sec=33447&di=7de4c89b4ec3ed81&bdenc=1&nsrc=IlPT2AEptyoA_yixCFOxXnANedT62v3IEQGG_yRJBXSxokDyqRLhGNVoXzb62Sm5HoCb9jHRsxw1wXSaAT-il17","keyword":"","browser":98,"language":37,"screen_color":33,"screen_size":0,"system":13,"platform":62,"operator":2,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":34021738,"uuid":"b00ae536-4ca5-474b-ba2b-fddd4c0b9674","ip":"1879278394","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/53236367.html","engine":0,"referer":"http:\/\/www.ijntv.cn\/inews\/53236367.html","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":2,"country":1,"province":16,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50017547,"uuid":"a76935f7-1da9-4dca-b95f-ba0f298ff5f5","ip":"1972898818","site":0,"source":0,"address":"http:\/\/www.dongfeng.net\/make\/6QRkYJX037Pkyw1co6-DPw.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":2,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539532801,"create_date":"2018-10-15 00:00:01"}

{"uid":50010757,"uuid":"dda000f3-930d-4e2a-8834-7edd05230574","ip":"1018017398","site":0,"source":0,"address":"http:\/\/e.hbqnb.com\/inews\/53383689.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":3,"city":45,"flash":"0","java":"0","request_time":1539532801,"create_date":"2018-10-15 00:00:01"}

{"uid":34004741,"uuid":"ef59dadf-fb58-46fa-aaf1-c1eb8a57e873","ip":"3071113907","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/55373856.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":6,"city":77,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

{"uid":50010336,"uuid":"65b0ac4f-2172-4746-a82e-fc8240240755","ip":"1996541572","site":0,"source":0,"address":"http:\/\/e.hbqnb.com\/inews\/53402956.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":0,"country":1,"province":8,"city":0,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

{"uid":34050322,"uuid":"f9dd0bda-5416-4535-b08b-d91c196977af","ip":"1962504456","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/5TJU_3pIEFfcni314q80ZA.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":30,"city":0,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

[root@a ~]# tail /data/visitlog/visit-2018-10-15

{"uid":50001890,"uuid":"d1ed8885-280c-4f5d-a06b-112ac2c78b01","ip":"1909901422","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/grFFl7D-XfwzTXUuHIyFiA.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":0,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539619195,"create_date":"2018-10-15 23:59:55"}

{"uid":50005483,"uuid":"6dcf7b2a-dcd9-460f-8213-0dbe220d474c","ip":"974469123","site":0,"source":0,"address":"http:\/\/www.xichu.net\/look\/45101554.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":3,"country":1,"province":212,"city":0,"flash":"0","java":"0","request_time":1539619196,"create_date":"2018-10-15 23:59:56"}

{"uid":50013267,"uuid":"46245679-8699-48ff-abe2-f691d3f14aa0","ip":"1943015812","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/PMekyfhQyRj8Qzu81ZD6Ww.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":31,"city":384,"flash":"0","java":"0","request_time":1539619197,"create_date":"2018-10-15 23:59:57"}

{"uid":50009806,"uuid":"75d15fb3-8f90-409c-ae16-f6dd58b83979","ip":"3733153632","site":0,"source":0,"address":"http:\/\/www.metalnews.cn\/xinxi\/xlOGEPlObiXPgZGG1ZTFxg.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":0,"country":1,"province":2,"city":0,"flash":"0","java":"0","request_time":1539619197,"create_date":"2018-10-15 23:59:57"}

{"uid":50015377,"uuid":"e70a8aaa-6798-4a49-a3ef-93ff0f6b875c","ip":"2090975143","site":0,"source":0,"address":"http:\/\/www.zjkonline.com\/well\/ycEzO-8D4LBWdiCA6kjb2w.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":3,"country":1,"province":26,"city":342,"flash":"0","java":"0","request_time":1539619198,"create_date":"2018-10-15 23:59:58"}

{"uid":50014416,"uuid":"e7441c1d-f40e-4851-9220-58ed03a416db","ip":"3060659990","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/HYSAdHafvcC5TeNfN_Q9fQ.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":17,"city":0,"flash":"0","java":"0","request_time":1539619198,"create_date":"2018-10-15 23:59:58"}

{"uid":50016615,"uuid":"32185523-813f-42b2-aca6-5fb03d0a71a0","ip":"2008640953","site":0,"source":0,"address":"http:\/\/www.zjkonline.com\/well\/QToFaPQVeQ9ix2EuIAp03Q.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":3,"country":1,"province":22,"city":293,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":34032314,"uuid":"315ea4eb-8fa1-4430-a38f-bcbeba450897","ip":"827363088","site":0,"source":0,"address":"http:\/\/www.01hr.com\/bfdx\/45995002.html","engine":0,"referer":"","keyword":"","browser":5,"language":0,"screen_color":34,"screen_size":0,"system":67,"platform":60,"operator":1,"country":1,"province":16,"city":226,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":34032314,"uuid":"315ea4eb-8fa1-4430-a38f-bcbeba450897","ip":"3026902583","site":0,"source":0,"address":"http:\/\/www.01hr.com\/bfdx\/45995002.html","engine":0,"referer":"http:\/\/www.01hr.com\/bfdx\/45995002.html","keyword":"","browser":5,"language":0,"screen_color":34,"screen_size":0,"system":67,"platform":60,"operator":1,"country":1,"province":16,"city":221,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":50008572,"uuid":"0fa00371-f97a-429f-bfdd-5bdfdb5b6f92","ip":"3747928108","site":0,"source":0,"address":"http:\/\/www.xxcmw.com\/shangxun\/55410287.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":2,"country":1,"province":18,"city":0,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

[root@a ~]#

当前场景借助负载均衡，对网站的访问分发到3个web服务器，同时各自访客日志落入相应web服务器（日志没有做集中化处理）

原方案是借助日志上报，给阿里的日志服务，然后在投入到阿里的dataworks中计算，最后写入mysql

现方案借助开源组件，实现mapReduce过程，得到汇总数据；

同时提高实时性。

Powered by Flink的更多相关文章

腾讯大数据平台Oceanus: A one-stop platform for real time stream processing powered by Apache Flink
January 25, 2019Use Cases, Apache Flink The Big Data Team at Tencent In recent years, the increa ...
Flink应用案例：How Trackunit leverages Flink to process real-time data from industrial IoT devices
January 22, 2019Use Cases, Apache Flink Lasse Nedergaard Recently there has been significant dis ...
Flink写入kafka时，只写入kafka的部分Partitioner，无法写所有的Partitioner问题
1. 写在前面在利用flink实时计算的时候,往往会从kafka读取数据写入数据到kafka,但会发现当kafka多个Partitioner时,特别在P量级数据为了kafka的性能kafka的节点有 ...
从零构建Flink SQL计算平台 - 1平台搭建
一.理想与现实 Apache Flink 是一个分布式流批一体化的开源平台.Flink 的核心是一个提供数据分发.通信以及自动容错的流计算引擎.Flink 在流计算之上构建批处理,并且原生的支持迭代计 ...
apache flink 入门
配置环境包括 JAVA_HOME jobmanager.rpc.address jobmanager.heap.mb 和 taskmanager.heap.mb taskmanager.number ...
Flink 1.1 – ResourceManager
Flink resource manager的作用如图, FlinkResourceManager /** * * <h1>Worker allocation steps</h1 ...
Apache Flink初接触
Apache Flink闻名已久,一直没有亲自尝试一把,这两天看了文档,发现在real-time streaming方面,Flink提供了更多高阶的实用函数. 用Apache Flink实现WordC ...
Flink - InstanceManager
InstanceManager用于管理JobManager申请到的taskManager和slots资源 /** * Simple manager that keeps track of which ...
Flink – window operator
参考, http://wuchong.me/blog/2016/05/25/flink-internals-window-mechanism/ http://wuchong.me/blog/201 ...

随机推荐

轻量集群管理工具PSSH
PSSH 的意思是 Parallel SSH,并行的SSH,很好理解,PSSH 可以让一条命令在多个服务器上同时执行这就简化了集群的管理工作,例如想查看一下各台服务器现在的负载状况,就可以通过 PS ...
Atitit. 单列索引与多列索引多个条件的查询原理与设计实现
Atitit. 单列索引与多列索引多个条件的查询原理与设计实现 1. MySQL只能使用一个索引1 1.1. 最左前缀1 1.2. 从另一方面理解,它相当于我们创建了(firstname,last ...
[git]git动画教程
git学习资料比较好的有廖雪峰的教程还有2个动画教程: https://www.zhihu.com/question/38008771  git-scm  廖雪峰-Git教程  git-for ...
Java序列化与反序列化学习（一）
一.序列化与反序列化概述当两个进程在进行远程通信时,彼此可以发送各种类型的数据.无论是何种类型的数据,都会以二进制序列的形式在网络上传送.发送方需要把这个Java对象转换为字节序列,才能在网 ...
李洪强iOS开发之iOS学习方法收集
李洪强iOS开发之iOS学习方法收集在这里收集一些iOS学习方法,会不断更新项目简述日期一年多iOS开发总结作者总结了自己一年多的iOS学习经验,对于iOS初学者来说很多地方是可以借鉴的 ...
Windows虚拟环境下安装mysql-python
因为在虚拟环境下安装mysql-python走了许多弯路,特此记录,也希望以后的朋友避免像我一样,被环境配置问题搞的半死直接使用pip安装mysql-python会报错 pip install My ...
linux学习笔记22---命令diff和diff3
diff 命令是 linux上非常重要的工具,用于比较文件的内容,特别是比较两个版本不同的文件以找到改动的地方.diff在命令行中打印每一个行的改动.最新版本的diff还支持二进制文件.diff程序的 ...
quick-cocos2d-x 创建自定义lua绑定c++类
内容主要参考 “在quick-cocos2d-x中添加自定义的类给lua使用” ( http://www.codeo4.cn/archives/746) 1. quick-coco2d-x 使用 to ...
ef6 code first with Oracle 试玩记录
对于oracle 使用code first 这边文章不错: http://www.cnblogs.com/wlflovenet/p/4187455.html 补充一下: 文章中提到machine.co ...
Flask 安装快速入门
$ pip install flask Flask自带的Server在端口5000上监听: ython app.py flask通过request.form['name']来获取表单的内容. 外部可见 ...

Powered by Flink

Powered by Flink

Powered by Flink的更多相关文章

随机推荐

热门专题