Apache Flink: Powered by Flink https://flink.apache.org/poweredby.html

Powered by Flink

Apache Flink powers business-critical applications in many companies and enterprises around the globe. On this page, we present a few notable Flink users that run interesting use cases in production and link to resources that discuss their applications in more detail.

More Flink users are listed in the Powered by Flink directory in the project wiki. Please note that the list is not comprehensive. We only add users that explicitly ask to be listed.

If you would you like to be included on this page, please reach out to the Flink user mailing list and let us know.

Alibaba, the world's largest retailer, uses a fork of Flink called Blink to optimize search rankings in real time.

Read more on the Zalando Tech Blog

访客日志分析

[root@a ~]# ll -ash /data/visitlog/
total 3.2G
20K drwxr-xr-x 2 nginx nginx 20K Oct 16 00:00 .
4.0K drwxr-xr-x 12 root root 4.0K Oct 12 09:41 ..
48M -rw-r--r-- 1 nginx nginx 48M Sep 18 23:59 visit-2018-09-18
66M -rw-r--r-- 1 nginx nginx 66M Sep 19 23:59 visit-2018-09-19
64M -rw-r--r-- 1 nginx nginx 64M Sep 20 23:59 visit-2018-09-20
68M -rw-r--r-- 1 nginx nginx 68M Sep 21 23:59 visit-2018-09-21
70M -rw-r--r-- 1 nginx nginx 70M Sep 22 23:59 visit-2018-09-22
63M -rw-r--r-- 1 nginx nginx 63M Sep 23 23:59 visit-2018-09-23
62M -rw-r--r-- 1 nginx nginx 62M Sep 24 23:59 visit-2018-09-24
70M -rw-r--r-- 1 nginx nginx 70M Sep 25 23:59 visit-2018-09-25
74M -rw-r--r-- 1 nginx nginx 74M Sep 26 23:59 visit-2018-09-26
78M -rw-r--r-- 1 nginx nginx 78M Sep 27 23:59 visit-2018-09-27
102M -rw-r--r-- 1 nginx nginx 102M Sep 28 23:59 visit-2018-09-28
125M -rw-r--r-- 1 nginx nginx 125M Sep 29 23:59 visit-2018-09-29
123M -rw-r--r-- 1 nginx nginx 123M Sep 30 23:59 visit-2018-09-30
114M -rw-r--r-- 1 nginx nginx 114M Oct 1 23:59 visit-2018-10-01
112M -rw-r--r-- 1 nginx nginx 112M Oct 2 23:59 visit-2018-10-02
120M -rw-r--r-- 1 nginx nginx 120M Oct 3 23:59 visit-2018-10-03
119M -rw-r--r-- 1 nginx nginx 119M Oct 4 23:59 visit-2018-10-04
128M -rw-r--r-- 1 nginx nginx 128M Oct 5 23:59 visit-2018-10-05
126M -rw-r--r-- 1 nginx nginx 126M Oct 6 23:59 visit-2018-10-06
124M -rw-r--r-- 1 nginx nginx 124M Oct 7 23:59 visit-2018-10-07
141M -rw-r--r-- 1 nginx nginx 141M Oct 8 23:59 visit-2018-10-08
141M -rw-r--r-- 1 nginx nginx 141M Oct 9 23:59 visit-2018-10-09
146M -rw-r--r-- 1 nginx nginx 146M Oct 10 23:59 visit-2018-10-10
137M -rw-r--r-- 1 nginx nginx 137M Oct 11 23:59 visit-2018-10-11
144M -rw-r--r-- 1 nginx nginx 144M Oct 12 23:59 visit-2018-10-12
139M -rw-r--r-- 1 nginx nginx 139M Oct 13 23:59 visit-2018-10-13
196M -rw-r--r-- 1 nginx nginx 196M Oct 14 23:59 visit-2018-10-14
245M -rw-r--r-- 1 nginx nginx 245M Oct 15 23:59 visit-2018-10-15
119M -rw-r--r-- 1 nginx nginx 119M Oct 16 11:40 visit-2018-10-16
[root@a ~]#

[root@a ~]# head /data/visitlog/visit-2018-10-15

{"uid":50007901,"uuid":"61408c43-b4f5-4614-86a8-aebbc3943820","ip":"3071402833","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/ZxJClxN76LzgYWJD-vQAIA.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":6,"city":77,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50017122,"uuid":"4fe41ece-e28a-4ba1-871e-8dfecbfc1d25","ip":"2004914835","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/S5vPXX4IK3PJQzipD_JbRg.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":6,"city":79,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50012077,"uuid":"28d1c40b-3d01-47bb-9532-abd39b2e1600","ip":"608159022","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/u2JljIAn5juvLng5oaIx8Q.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":3,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":34056498,"uuid":"bf0d9674-8835-497d-88eb-75b1a12a88fb","ip":"1971867257","site":70,"source":0,"address":"http:\/\/www.jqw.com\/bianmin\/m\/33239101.html","engine":2,"referer":"https:\/\/m.baidu.com\/from=1014517c\/bd_page_type=1\/ssid=0\/uid=0\/pu=sz%401320_480%2Ccuid%40_u20tguaHugMu-iqYa-3igPq-ilAaSi10827ag8m28KaLqqqB%2Ccua%40_avLC_aE2f4qywoUfpw1z4aS2N_VaXiHoavRhGQHNqqqB%2Ccut%40lfSpI0tI2iyjaXiDyuvGC5kcSMzWtqqSB%2Cosname%40baiduboxapp%2Cctv%402%2Ccfrom%401014517c%2Ccen%40cuid_cua_cut%2Ccsrc%40bdbox_tserch_txt%2Cta%40zbios_2_6.0_6_10.10%2Cusm%402%2Cvmgdb%400020100228y\/baiduid=1CC91FA18C1983E5C17C8EEF12E0C62E\/w=0_10_\/t=zbios\/l=1\/tc?ref=www_zbios&lid=8867811778563143063&order=2&fm=alop&waplogo=1&is_baidu=0&h5ad=0&tj=www_normal_2_0_10_title&vit=osres&waput=3&cltj=normal_title&asres=1&title=%E9%9D%92%E5%B2%9B%E5%A4%96%E5%A2%99%E4%BF%9D%E6%B8%A9%E6%96%BD%E5%B7%A5%2C%E9%98%B2%E6%B0%B4%E6%96%BD%E5%B7%A5%E4%B8%93%E4%B8%9A%E4%BB%8E%E4%BA%8B%E5%BB%BA%E7%AD%91%E5%8A%A0%E5%9B%BA-%E9%87%91%E6%B3%89%E7%BD%91&hwj=1603686807417560&dict=-1&wd=&eqid=7b10cbd43447f400100000015bc367c5&w_qd=IlPT2AEptyoA_ykwsPIcyRexDE6OlY9oqjwYlBPVtg5rXvUzNE-dyJzfkxJ7m_oU5xJuDcK&tcplug=1&sec=33447&di=7de4c89b4ec3ed81&bdenc=1&nsrc=IlPT2AEptyoA_yixCFOxXnANedT62v3IEQGG_yRJBXSxokDyqRLhGNVoXzb62Sm5HoCb9jHRsxw1wXSaAT-il17","keyword":"","browser":98,"language":37,"screen_color":33,"screen_size":0,"system":13,"platform":62,"operator":2,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":34021738,"uuid":"b00ae536-4ca5-474b-ba2b-fddd4c0b9674","ip":"1879278394","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/53236367.html","engine":0,"referer":"http:\/\/www.ijntv.cn\/inews\/53236367.html","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":2,"country":1,"province":16,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50017547,"uuid":"a76935f7-1da9-4dca-b95f-ba0f298ff5f5","ip":"1972898818","site":0,"source":0,"address":"http:\/\/www.dongfeng.net\/make\/6QRkYJX037Pkyw1co6-DPw.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":2,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539532801,"create_date":"2018-10-15 00:00:01"}

{"uid":50010757,"uuid":"dda000f3-930d-4e2a-8834-7edd05230574","ip":"1018017398","site":0,"source":0,"address":"http:\/\/e.hbqnb.com\/inews\/53383689.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":3,"city":45,"flash":"0","java":"0","request_time":1539532801,"create_date":"2018-10-15 00:00:01"}

{"uid":34004741,"uuid":"ef59dadf-fb58-46fa-aaf1-c1eb8a57e873","ip":"3071113907","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/55373856.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":6,"city":77,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

{"uid":50010336,"uuid":"65b0ac4f-2172-4746-a82e-fc8240240755","ip":"1996541572","site":0,"source":0,"address":"http:\/\/e.hbqnb.com\/inews\/53402956.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":0,"country":1,"province":8,"city":0,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

{"uid":34050322,"uuid":"f9dd0bda-5416-4535-b08b-d91c196977af","ip":"1962504456","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/5TJU_3pIEFfcni314q80ZA.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":30,"city":0,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

[root@a ~]# tail /data/visitlog/visit-2018-10-15

{"uid":50001890,"uuid":"d1ed8885-280c-4f5d-a06b-112ac2c78b01","ip":"1909901422","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/grFFl7D-XfwzTXUuHIyFiA.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":0,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539619195,"create_date":"2018-10-15 23:59:55"}

{"uid":50005483,"uuid":"6dcf7b2a-dcd9-460f-8213-0dbe220d474c","ip":"974469123","site":0,"source":0,"address":"http:\/\/www.xichu.net\/look\/45101554.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":3,"country":1,"province":212,"city":0,"flash":"0","java":"0","request_time":1539619196,"create_date":"2018-10-15 23:59:56"}

{"uid":50013267,"uuid":"46245679-8699-48ff-abe2-f691d3f14aa0","ip":"1943015812","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/PMekyfhQyRj8Qzu81ZD6Ww.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":31,"city":384,"flash":"0","java":"0","request_time":1539619197,"create_date":"2018-10-15 23:59:57"}

{"uid":50009806,"uuid":"75d15fb3-8f90-409c-ae16-f6dd58b83979","ip":"3733153632","site":0,"source":0,"address":"http:\/\/www.metalnews.cn\/xinxi\/xlOGEPlObiXPgZGG1ZTFxg.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":0,"country":1,"province":2,"city":0,"flash":"0","java":"0","request_time":1539619197,"create_date":"2018-10-15 23:59:57"}

{"uid":50015377,"uuid":"e70a8aaa-6798-4a49-a3ef-93ff0f6b875c","ip":"2090975143","site":0,"source":0,"address":"http:\/\/www.zjkonline.com\/well\/ycEzO-8D4LBWdiCA6kjb2w.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":3,"country":1,"province":26,"city":342,"flash":"0","java":"0","request_time":1539619198,"create_date":"2018-10-15 23:59:58"}

{"uid":50014416,"uuid":"e7441c1d-f40e-4851-9220-58ed03a416db","ip":"3060659990","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/HYSAdHafvcC5TeNfN_Q9fQ.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":17,"city":0,"flash":"0","java":"0","request_time":1539619198,"create_date":"2018-10-15 23:59:58"}

{"uid":50016615,"uuid":"32185523-813f-42b2-aca6-5fb03d0a71a0","ip":"2008640953","site":0,"source":0,"address":"http:\/\/www.zjkonline.com\/well\/QToFaPQVeQ9ix2EuIAp03Q.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":3,"country":1,"province":22,"city":293,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":34032314,"uuid":"315ea4eb-8fa1-4430-a38f-bcbeba450897","ip":"827363088","site":0,"source":0,"address":"http:\/\/www.01hr.com\/bfdx\/45995002.html","engine":0,"referer":"","keyword":"","browser":5,"language":0,"screen_color":34,"screen_size":0,"system":67,"platform":60,"operator":1,"country":1,"province":16,"city":226,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":34032314,"uuid":"315ea4eb-8fa1-4430-a38f-bcbeba450897","ip":"3026902583","site":0,"source":0,"address":"http:\/\/www.01hr.com\/bfdx\/45995002.html","engine":0,"referer":"http:\/\/www.01hr.com\/bfdx\/45995002.html","keyword":"","browser":5,"language":0,"screen_color":34,"screen_size":0,"system":67,"platform":60,"operator":1,"country":1,"province":16,"city":221,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":50008572,"uuid":"0fa00371-f97a-429f-bfdd-5bdfdb5b6f92","ip":"3747928108","site":0,"source":0,"address":"http:\/\/www.xxcmw.com\/shangxun\/55410287.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":2,"country":1,"province":18,"city":0,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

[root@a ~]#

当前场景借助负载均衡，对网站的访问分发到3个web服务器，同时各自访客日志落入相应web服务器（日志没有做集中化处理）

原方案是借助日志上报，给阿里的日志服务，然后在投入到阿里的dataworks中计算，最后写入mysql

现方案借助开源组件，实现mapReduce过程，得到汇总数据；

同时提高实时性。

Powered by Flink的更多相关文章

腾讯大数据平台Oceanus: A one-stop platform for real time stream processing powered by Apache Flink
January 25, 2019Use Cases, Apache Flink The Big Data Team at Tencent In recent years, the increa ...
Flink应用案例：How Trackunit leverages Flink to process real-time data from industrial IoT devices
January 22, 2019Use Cases, Apache Flink Lasse Nedergaard Recently there has been significant dis ...
Flink写入kafka时，只写入kafka的部分Partitioner，无法写所有的Partitioner问题
1. 写在前面在利用flink实时计算的时候,往往会从kafka读取数据写入数据到kafka,但会发现当kafka多个Partitioner时,特别在P量级数据为了kafka的性能kafka的节点有 ...
从零构建Flink SQL计算平台 - 1平台搭建
一.理想与现实 Apache Flink 是一个分布式流批一体化的开源平台.Flink 的核心是一个提供数据分发.通信以及自动容错的流计算引擎.Flink 在流计算之上构建批处理,并且原生的支持迭代计 ...
apache flink 入门
配置环境包括 JAVA_HOME jobmanager.rpc.address jobmanager.heap.mb 和 taskmanager.heap.mb taskmanager.number ...
Flink 1.1 – ResourceManager
Flink resource manager的作用如图, FlinkResourceManager /** * * <h1>Worker allocation steps</h1 ...
Apache Flink初接触
Apache Flink闻名已久,一直没有亲自尝试一把,这两天看了文档,发现在real-time streaming方面,Flink提供了更多高阶的实用函数. 用Apache Flink实现WordC ...
Flink - InstanceManager
InstanceManager用于管理JobManager申请到的taskManager和slots资源 /** * Simple manager that keeps track of which ...
Flink – window operator
参考, http://wuchong.me/blog/2016/05/25/flink-internals-window-mechanism/ http://wuchong.me/blog/201 ...

随机推荐

zookeeper（三）：java操作zookeeper
引入jar包首先要使用java操作zookeeper,zookeeper的javaclient 使我们更轻松的去对zookeeper进行各种操作,我们引入zookeeper-3.4.5.jar 和 ...
分享分享JavaScript100例-仅供参考
最近一直在做项目,分享下以前收集的Javascript100例,仅供参考. http://files.cnblogs.com/52net/JavaScript100例.zip
Unity3D_NGUI_性能优化实践_CPU卡顿
http://gad.qq.com/college/articledetail/7083468 博尔特以9.58秒创造了百米世界纪录,假设他是跑酷游戏的角色,卡顿一帧就足以把冠军拱手让人. Unity ...
ArcGIS教程：Geostatistical Analyst 应用演示样例
Geostatistical Analyst 满足多种不同应用的需求. 下面是 Geostatistical Analyst 的一小部分应用演示样例. 探索性空间数据分析 Geostatistical ...
解决Jenkins无法编译Egret5.0项目的问题
问题的原因可查看:https://blog.csdn.net/sujun10/article/details/75512929 解决造成这个问题的原因是用户权限分配,你可以通过下面几步解决,而非改e ...
转-subl配置全栈开发环境
为 Sublime Text 3 设置 Python 的全栈开发环境 Sublime Text 3 (ST3) 是一个轻量级的跨平台文字编辑器,尤以其轻快的速度,易用性和强大的社区支持而著称.它一经面 ...
NPOI 导出Excel图片（网络）
导出网络图片,需要将网络图片下载到本地或者内存流中,建议下载的时候使用缩略图: 高清图片效率慢: Uri uri = new Uri(imgPath); //imgPath :网络图片地址 WebRe ...
Activiti(一)--安装配置具体解释
有一段时间没有更新文章了,尽管有一直在写文章,但是一直没有更新到博客内,这段时间写的文章大多还是以技术为主. 接下来的系列文章将会来讨论企业工作流的开发,主要是来研究开源工作流Activiti的使用. ...
机器学习之svm---cv wiki svm
http://www.opencv.org.cn/opencvdoc/2.3.2/html/doc/tutorials/ml/introduction_to_svm/introduction_to_s ...
使用Using的注意事项
参数传递 C#中有四种参数类型:值类型,Ref参数,Out参数,params参数.默认参数都是以传值方式传递,这意味着方法中的变量会在内存中被分配新的存储空间,并赋值.对于引用类型,这种传值意味着传递 ...

Powered by Flink

Powered by Flink

Powered by Flink的更多相关文章

随机推荐

热门专题