Apache Flink: Powered by Flink https://flink.apache.org/poweredby.html

Powered by Flink

Apache Flink powers business-critical applications in many companies and enterprises around the globe. On this page, we present a few notable Flink users that run interesting use cases in production and link to resources that discuss their applications in more detail.

More Flink users are listed in the Powered by Flink directory in the project wiki. Please note that the list is not comprehensive. We only add users that explicitly ask to be listed.

If you would you like to be included on this page, please reach out to the Flink user mailing list and let us know.

Alibaba, the world's largest retailer, uses a fork of Flink called Blink to optimize search rankings in real time.

Read more on the Zalando Tech Blog

访客日志分析

[root@a ~]# ll -ash /data/visitlog/
total 3.2G
20K drwxr-xr-x 2 nginx nginx 20K Oct 16 00:00 .
4.0K drwxr-xr-x 12 root root 4.0K Oct 12 09:41 ..
48M -rw-r--r-- 1 nginx nginx 48M Sep 18 23:59 visit-2018-09-18
66M -rw-r--r-- 1 nginx nginx 66M Sep 19 23:59 visit-2018-09-19
64M -rw-r--r-- 1 nginx nginx 64M Sep 20 23:59 visit-2018-09-20
68M -rw-r--r-- 1 nginx nginx 68M Sep 21 23:59 visit-2018-09-21
70M -rw-r--r-- 1 nginx nginx 70M Sep 22 23:59 visit-2018-09-22
63M -rw-r--r-- 1 nginx nginx 63M Sep 23 23:59 visit-2018-09-23
62M -rw-r--r-- 1 nginx nginx 62M Sep 24 23:59 visit-2018-09-24
70M -rw-r--r-- 1 nginx nginx 70M Sep 25 23:59 visit-2018-09-25
74M -rw-r--r-- 1 nginx nginx 74M Sep 26 23:59 visit-2018-09-26
78M -rw-r--r-- 1 nginx nginx 78M Sep 27 23:59 visit-2018-09-27
102M -rw-r--r-- 1 nginx nginx 102M Sep 28 23:59 visit-2018-09-28
125M -rw-r--r-- 1 nginx nginx 125M Sep 29 23:59 visit-2018-09-29
123M -rw-r--r-- 1 nginx nginx 123M Sep 30 23:59 visit-2018-09-30
114M -rw-r--r-- 1 nginx nginx 114M Oct 1 23:59 visit-2018-10-01
112M -rw-r--r-- 1 nginx nginx 112M Oct 2 23:59 visit-2018-10-02
120M -rw-r--r-- 1 nginx nginx 120M Oct 3 23:59 visit-2018-10-03
119M -rw-r--r-- 1 nginx nginx 119M Oct 4 23:59 visit-2018-10-04
128M -rw-r--r-- 1 nginx nginx 128M Oct 5 23:59 visit-2018-10-05
126M -rw-r--r-- 1 nginx nginx 126M Oct 6 23:59 visit-2018-10-06
124M -rw-r--r-- 1 nginx nginx 124M Oct 7 23:59 visit-2018-10-07
141M -rw-r--r-- 1 nginx nginx 141M Oct 8 23:59 visit-2018-10-08
141M -rw-r--r-- 1 nginx nginx 141M Oct 9 23:59 visit-2018-10-09
146M -rw-r--r-- 1 nginx nginx 146M Oct 10 23:59 visit-2018-10-10
137M -rw-r--r-- 1 nginx nginx 137M Oct 11 23:59 visit-2018-10-11
144M -rw-r--r-- 1 nginx nginx 144M Oct 12 23:59 visit-2018-10-12
139M -rw-r--r-- 1 nginx nginx 139M Oct 13 23:59 visit-2018-10-13
196M -rw-r--r-- 1 nginx nginx 196M Oct 14 23:59 visit-2018-10-14
245M -rw-r--r-- 1 nginx nginx 245M Oct 15 23:59 visit-2018-10-15
119M -rw-r--r-- 1 nginx nginx 119M Oct 16 11:40 visit-2018-10-16
[root@a ~]#

[root@a ~]# head /data/visitlog/visit-2018-10-15

{"uid":50007901,"uuid":"61408c43-b4f5-4614-86a8-aebbc3943820","ip":"3071402833","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/ZxJClxN76LzgYWJD-vQAIA.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":6,"city":77,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50017122,"uuid":"4fe41ece-e28a-4ba1-871e-8dfecbfc1d25","ip":"2004914835","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/S5vPXX4IK3PJQzipD_JbRg.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":6,"city":79,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50012077,"uuid":"28d1c40b-3d01-47bb-9532-abd39b2e1600","ip":"608159022","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/u2JljIAn5juvLng5oaIx8Q.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":3,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":34056498,"uuid":"bf0d9674-8835-497d-88eb-75b1a12a88fb","ip":"1971867257","site":70,"source":0,"address":"http:\/\/www.jqw.com\/bianmin\/m\/33239101.html","engine":2,"referer":"https:\/\/m.baidu.com\/from=1014517c\/bd_page_type=1\/ssid=0\/uid=0\/pu=sz%401320_480%2Ccuid%40_u20tguaHugMu-iqYa-3igPq-ilAaSi10827ag8m28KaLqqqB%2Ccua%40_avLC_aE2f4qywoUfpw1z4aS2N_VaXiHoavRhGQHNqqqB%2Ccut%40lfSpI0tI2iyjaXiDyuvGC5kcSMzWtqqSB%2Cosname%40baiduboxapp%2Cctv%402%2Ccfrom%401014517c%2Ccen%40cuid_cua_cut%2Ccsrc%40bdbox_tserch_txt%2Cta%40zbios_2_6.0_6_10.10%2Cusm%402%2Cvmgdb%400020100228y\/baiduid=1CC91FA18C1983E5C17C8EEF12E0C62E\/w=0_10_\/t=zbios\/l=1\/tc?ref=www_zbios&lid=8867811778563143063&order=2&fm=alop&waplogo=1&is_baidu=0&h5ad=0&tj=www_normal_2_0_10_title&vit=osres&waput=3&cltj=normal_title&asres=1&title=%E9%9D%92%E5%B2%9B%E5%A4%96%E5%A2%99%E4%BF%9D%E6%B8%A9%E6%96%BD%E5%B7%A5%2C%E9%98%B2%E6%B0%B4%E6%96%BD%E5%B7%A5%E4%B8%93%E4%B8%9A%E4%BB%8E%E4%BA%8B%E5%BB%BA%E7%AD%91%E5%8A%A0%E5%9B%BA-%E9%87%91%E6%B3%89%E7%BD%91&hwj=1603686807417560&dict=-1&wd=&eqid=7b10cbd43447f400100000015bc367c5&w_qd=IlPT2AEptyoA_ykwsPIcyRexDE6OlY9oqjwYlBPVtg5rXvUzNE-dyJzfkxJ7m_oU5xJuDcK&tcplug=1&sec=33447&di=7de4c89b4ec3ed81&bdenc=1&nsrc=IlPT2AEptyoA_yixCFOxXnANedT62v3IEQGG_yRJBXSxokDyqRLhGNVoXzb62Sm5HoCb9jHRsxw1wXSaAT-il17","keyword":"","browser":98,"language":37,"screen_color":33,"screen_size":0,"system":13,"platform":62,"operator":2,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":34021738,"uuid":"b00ae536-4ca5-474b-ba2b-fddd4c0b9674","ip":"1879278394","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/53236367.html","engine":0,"referer":"http:\/\/www.ijntv.cn\/inews\/53236367.html","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":2,"country":1,"province":16,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50017547,"uuid":"a76935f7-1da9-4dca-b95f-ba0f298ff5f5","ip":"1972898818","site":0,"source":0,"address":"http:\/\/www.dongfeng.net\/make\/6QRkYJX037Pkyw1co6-DPw.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":2,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539532801,"create_date":"2018-10-15 00:00:01"}

{"uid":50010757,"uuid":"dda000f3-930d-4e2a-8834-7edd05230574","ip":"1018017398","site":0,"source":0,"address":"http:\/\/e.hbqnb.com\/inews\/53383689.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":3,"city":45,"flash":"0","java":"0","request_time":1539532801,"create_date":"2018-10-15 00:00:01"}

{"uid":34004741,"uuid":"ef59dadf-fb58-46fa-aaf1-c1eb8a57e873","ip":"3071113907","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/55373856.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":6,"city":77,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

{"uid":50010336,"uuid":"65b0ac4f-2172-4746-a82e-fc8240240755","ip":"1996541572","site":0,"source":0,"address":"http:\/\/e.hbqnb.com\/inews\/53402956.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":0,"country":1,"province":8,"city":0,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

{"uid":34050322,"uuid":"f9dd0bda-5416-4535-b08b-d91c196977af","ip":"1962504456","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/5TJU_3pIEFfcni314q80ZA.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":30,"city":0,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

[root@a ~]# tail /data/visitlog/visit-2018-10-15

{"uid":50001890,"uuid":"d1ed8885-280c-4f5d-a06b-112ac2c78b01","ip":"1909901422","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/grFFl7D-XfwzTXUuHIyFiA.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":0,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539619195,"create_date":"2018-10-15 23:59:55"}

{"uid":50005483,"uuid":"6dcf7b2a-dcd9-460f-8213-0dbe220d474c","ip":"974469123","site":0,"source":0,"address":"http:\/\/www.xichu.net\/look\/45101554.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":3,"country":1,"province":212,"city":0,"flash":"0","java":"0","request_time":1539619196,"create_date":"2018-10-15 23:59:56"}

{"uid":50013267,"uuid":"46245679-8699-48ff-abe2-f691d3f14aa0","ip":"1943015812","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/PMekyfhQyRj8Qzu81ZD6Ww.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":31,"city":384,"flash":"0","java":"0","request_time":1539619197,"create_date":"2018-10-15 23:59:57"}

{"uid":50009806,"uuid":"75d15fb3-8f90-409c-ae16-f6dd58b83979","ip":"3733153632","site":0,"source":0,"address":"http:\/\/www.metalnews.cn\/xinxi\/xlOGEPlObiXPgZGG1ZTFxg.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":0,"country":1,"province":2,"city":0,"flash":"0","java":"0","request_time":1539619197,"create_date":"2018-10-15 23:59:57"}

{"uid":50015377,"uuid":"e70a8aaa-6798-4a49-a3ef-93ff0f6b875c","ip":"2090975143","site":0,"source":0,"address":"http:\/\/www.zjkonline.com\/well\/ycEzO-8D4LBWdiCA6kjb2w.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":3,"country":1,"province":26,"city":342,"flash":"0","java":"0","request_time":1539619198,"create_date":"2018-10-15 23:59:58"}

{"uid":50014416,"uuid":"e7441c1d-f40e-4851-9220-58ed03a416db","ip":"3060659990","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/HYSAdHafvcC5TeNfN_Q9fQ.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":17,"city":0,"flash":"0","java":"0","request_time":1539619198,"create_date":"2018-10-15 23:59:58"}

{"uid":50016615,"uuid":"32185523-813f-42b2-aca6-5fb03d0a71a0","ip":"2008640953","site":0,"source":0,"address":"http:\/\/www.zjkonline.com\/well\/QToFaPQVeQ9ix2EuIAp03Q.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":3,"country":1,"province":22,"city":293,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":34032314,"uuid":"315ea4eb-8fa1-4430-a38f-bcbeba450897","ip":"827363088","site":0,"source":0,"address":"http:\/\/www.01hr.com\/bfdx\/45995002.html","engine":0,"referer":"","keyword":"","browser":5,"language":0,"screen_color":34,"screen_size":0,"system":67,"platform":60,"operator":1,"country":1,"province":16,"city":226,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":34032314,"uuid":"315ea4eb-8fa1-4430-a38f-bcbeba450897","ip":"3026902583","site":0,"source":0,"address":"http:\/\/www.01hr.com\/bfdx\/45995002.html","engine":0,"referer":"http:\/\/www.01hr.com\/bfdx\/45995002.html","keyword":"","browser":5,"language":0,"screen_color":34,"screen_size":0,"system":67,"platform":60,"operator":1,"country":1,"province":16,"city":221,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":50008572,"uuid":"0fa00371-f97a-429f-bfdd-5bdfdb5b6f92","ip":"3747928108","site":0,"source":0,"address":"http:\/\/www.xxcmw.com\/shangxun\/55410287.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":2,"country":1,"province":18,"city":0,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

[root@a ~]#

当前场景借助负载均衡，对网站的访问分发到3个web服务器，同时各自访客日志落入相应web服务器（日志没有做集中化处理）

原方案是借助日志上报，给阿里的日志服务，然后在投入到阿里的dataworks中计算，最后写入mysql

现方案借助开源组件，实现mapReduce过程，得到汇总数据；

同时提高实时性。

Powered by Flink的更多相关文章

腾讯大数据平台Oceanus: A one-stop platform for real time stream processing powered by Apache Flink
January 25, 2019Use Cases, Apache Flink The Big Data Team at Tencent In recent years, the increa ...
Flink应用案例：How Trackunit leverages Flink to process real-time data from industrial IoT devices
January 22, 2019Use Cases, Apache Flink Lasse Nedergaard Recently there has been significant dis ...
Flink写入kafka时，只写入kafka的部分Partitioner，无法写所有的Partitioner问题
1. 写在前面在利用flink实时计算的时候,往往会从kafka读取数据写入数据到kafka,但会发现当kafka多个Partitioner时,特别在P量级数据为了kafka的性能kafka的节点有 ...
从零构建Flink SQL计算平台 - 1平台搭建
一.理想与现实 Apache Flink 是一个分布式流批一体化的开源平台.Flink 的核心是一个提供数据分发.通信以及自动容错的流计算引擎.Flink 在流计算之上构建批处理,并且原生的支持迭代计 ...
apache flink 入门
配置环境包括 JAVA_HOME jobmanager.rpc.address jobmanager.heap.mb 和 taskmanager.heap.mb taskmanager.number ...
Flink 1.1 – ResourceManager
Flink resource manager的作用如图, FlinkResourceManager /** * * <h1>Worker allocation steps</h1 ...
Apache Flink初接触
Apache Flink闻名已久,一直没有亲自尝试一把,这两天看了文档,发现在real-time streaming方面,Flink提供了更多高阶的实用函数. 用Apache Flink实现WordC ...
Flink - InstanceManager
InstanceManager用于管理JobManager申请到的taskManager和slots资源 /** * Simple manager that keeps track of which ...
Flink – window operator
参考, http://wuchong.me/blog/2016/05/25/flink-internals-window-mechanism/ http://wuchong.me/blog/201 ...

随机推荐

window.print
function contentPrint(contentArea) { var initBody = document.body.innerHTML; if ($("div"). ...
Sublime Text 2 和 Verilog HDL
Sublime Text 2 和 Verilog HDL Date Fri 04 July 2014 Tags Sublime Text / Vivado Sublime Text 代码编辑器之于程 ...
热烈祝贺阿尔法Go首战告捷
这是人类的一大杰作和进步.一个国家和民族的未来在科技,靠造房子是成不了科技强国的. 当然,也要祝贺一下北上深房价突破历史高位.这也是伟大而不朽的成果.
李洪强IOS开发之iOS好项目收集
李洪强IOS开发之iOS好项目收集在这里收集一些最近出现的比较实用好玩的框架或者项目,会不断更新项目简述日期 SCTableViewCell 类似与QQ侧滑删除Cell的Demo 201501 ...
struts解决form提交的中文参数乱码问题
根据struts的工作原理,原文摘自<Java Web 开发实战经典> 在运行一个JSP页面前,会调用指定的ActionForm中的reset()方法,进行表单元素的初始化因此,在相应的 ...
jdbc.properties
#privilege database privilege.jdbc.driverClassName=com.mysql.jdbc.Driver privilege.jdbc.url=jdbc\:my ...
提高.net程序性能和稳定性-CLR Profile
CLR Profile能够看到应用程序的内存堆栈情况并且能够查询垃圾回收机制的行为.利用CLR Profile可以确定你的代码哪儿分配了太多内存,从而导致垃圾回收机制的执行,哪些代码长时间的占有内存. ...
Solr学习02：搭建Solr环境
一.安装虚拟机 Solr 必须运行在Java1.6 或更高版本的Java 虚拟机中,运行标准Solr 服务只需要安装JRE 即可,但如果需要扩展功能或编译源码则需要下载JDK 来完成.可以通过下面的地 ...
C扩展php的方法（制作php扩展库）
用PHP调用C扩展整个配置过程在CentOS下今天终于把C扩展加入到PHP中了,并且可以调用,废话就不说了,看下文. 一.必须先要安装Apache和mysql,这两个的安装过程我就不说了. ...
hdu6007 Mr. Panda and Crystal 最短路+完全背包
/** 题目:hdu6007 Mr. Panda and Crystal 链接:http://acm.hdu.edu.cn/showproblem.php?pid=6007 题意:魔法师有m能量,有n ...

Powered by Flink

Powered by Flink

Powered by Flink的更多相关文章

随机推荐

热门专题