Apache Flink: Powered by Flink https://flink.apache.org/poweredby.html

Powered by Flink

Apache Flink powers business-critical applications in many companies and enterprises around the globe. On this page, we present a few notable Flink users that run interesting use cases in production and link to resources that discuss their applications in more detail.

More Flink users are listed in the Powered by Flink directory in the project wiki. Please note that the list is not comprehensive. We only add users that explicitly ask to be listed.

If you would you like to be included on this page, please reach out to the Flink user mailing list and let us know.

Alibaba, the world's largest retailer, uses a fork of Flink called Blink to optimize search rankings in real time.

Read more on the Zalando Tech Blog

访客日志分析

[root@a ~]# ll -ash /data/visitlog/
total 3.2G
20K drwxr-xr-x 2 nginx nginx 20K Oct 16 00:00 .
4.0K drwxr-xr-x 12 root root 4.0K Oct 12 09:41 ..
48M -rw-r--r-- 1 nginx nginx 48M Sep 18 23:59 visit-2018-09-18
66M -rw-r--r-- 1 nginx nginx 66M Sep 19 23:59 visit-2018-09-19
64M -rw-r--r-- 1 nginx nginx 64M Sep 20 23:59 visit-2018-09-20
68M -rw-r--r-- 1 nginx nginx 68M Sep 21 23:59 visit-2018-09-21
70M -rw-r--r-- 1 nginx nginx 70M Sep 22 23:59 visit-2018-09-22
63M -rw-r--r-- 1 nginx nginx 63M Sep 23 23:59 visit-2018-09-23
62M -rw-r--r-- 1 nginx nginx 62M Sep 24 23:59 visit-2018-09-24
70M -rw-r--r-- 1 nginx nginx 70M Sep 25 23:59 visit-2018-09-25
74M -rw-r--r-- 1 nginx nginx 74M Sep 26 23:59 visit-2018-09-26
78M -rw-r--r-- 1 nginx nginx 78M Sep 27 23:59 visit-2018-09-27
102M -rw-r--r-- 1 nginx nginx 102M Sep 28 23:59 visit-2018-09-28
125M -rw-r--r-- 1 nginx nginx 125M Sep 29 23:59 visit-2018-09-29
123M -rw-r--r-- 1 nginx nginx 123M Sep 30 23:59 visit-2018-09-30
114M -rw-r--r-- 1 nginx nginx 114M Oct 1 23:59 visit-2018-10-01
112M -rw-r--r-- 1 nginx nginx 112M Oct 2 23:59 visit-2018-10-02
120M -rw-r--r-- 1 nginx nginx 120M Oct 3 23:59 visit-2018-10-03
119M -rw-r--r-- 1 nginx nginx 119M Oct 4 23:59 visit-2018-10-04
128M -rw-r--r-- 1 nginx nginx 128M Oct 5 23:59 visit-2018-10-05
126M -rw-r--r-- 1 nginx nginx 126M Oct 6 23:59 visit-2018-10-06
124M -rw-r--r-- 1 nginx nginx 124M Oct 7 23:59 visit-2018-10-07
141M -rw-r--r-- 1 nginx nginx 141M Oct 8 23:59 visit-2018-10-08
141M -rw-r--r-- 1 nginx nginx 141M Oct 9 23:59 visit-2018-10-09
146M -rw-r--r-- 1 nginx nginx 146M Oct 10 23:59 visit-2018-10-10
137M -rw-r--r-- 1 nginx nginx 137M Oct 11 23:59 visit-2018-10-11
144M -rw-r--r-- 1 nginx nginx 144M Oct 12 23:59 visit-2018-10-12
139M -rw-r--r-- 1 nginx nginx 139M Oct 13 23:59 visit-2018-10-13
196M -rw-r--r-- 1 nginx nginx 196M Oct 14 23:59 visit-2018-10-14
245M -rw-r--r-- 1 nginx nginx 245M Oct 15 23:59 visit-2018-10-15
119M -rw-r--r-- 1 nginx nginx 119M Oct 16 11:40 visit-2018-10-16
[root@a ~]#

[root@a ~]# head /data/visitlog/visit-2018-10-15

{"uid":50007901,"uuid":"61408c43-b4f5-4614-86a8-aebbc3943820","ip":"3071402833","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/ZxJClxN76LzgYWJD-vQAIA.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":6,"city":77,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50017122,"uuid":"4fe41ece-e28a-4ba1-871e-8dfecbfc1d25","ip":"2004914835","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/S5vPXX4IK3PJQzipD_JbRg.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":6,"city":79,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50012077,"uuid":"28d1c40b-3d01-47bb-9532-abd39b2e1600","ip":"608159022","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/u2JljIAn5juvLng5oaIx8Q.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":3,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":34056498,"uuid":"bf0d9674-8835-497d-88eb-75b1a12a88fb","ip":"1971867257","site":70,"source":0,"address":"http:\/\/www.jqw.com\/bianmin\/m\/33239101.html","engine":2,"referer":"https:\/\/m.baidu.com\/from=1014517c\/bd_page_type=1\/ssid=0\/uid=0\/pu=sz%401320_480%2Ccuid%40_u20tguaHugMu-iqYa-3igPq-ilAaSi10827ag8m28KaLqqqB%2Ccua%40_avLC_aE2f4qywoUfpw1z4aS2N_VaXiHoavRhGQHNqqqB%2Ccut%40lfSpI0tI2iyjaXiDyuvGC5kcSMzWtqqSB%2Cosname%40baiduboxapp%2Cctv%402%2Ccfrom%401014517c%2Ccen%40cuid_cua_cut%2Ccsrc%40bdbox_tserch_txt%2Cta%40zbios_2_6.0_6_10.10%2Cusm%402%2Cvmgdb%400020100228y\/baiduid=1CC91FA18C1983E5C17C8EEF12E0C62E\/w=0_10_\/t=zbios\/l=1\/tc?ref=www_zbios&lid=8867811778563143063&order=2&fm=alop&waplogo=1&is_baidu=0&h5ad=0&tj=www_normal_2_0_10_title&vit=osres&waput=3&cltj=normal_title&asres=1&title=%E9%9D%92%E5%B2%9B%E5%A4%96%E5%A2%99%E4%BF%9D%E6%B8%A9%E6%96%BD%E5%B7%A5%2C%E9%98%B2%E6%B0%B4%E6%96%BD%E5%B7%A5%E4%B8%93%E4%B8%9A%E4%BB%8E%E4%BA%8B%E5%BB%BA%E7%AD%91%E5%8A%A0%E5%9B%BA-%E9%87%91%E6%B3%89%E7%BD%91&hwj=1603686807417560&dict=-1&wd=&eqid=7b10cbd43447f400100000015bc367c5&w_qd=IlPT2AEptyoA_ykwsPIcyRexDE6OlY9oqjwYlBPVtg5rXvUzNE-dyJzfkxJ7m_oU5xJuDcK&tcplug=1&sec=33447&di=7de4c89b4ec3ed81&bdenc=1&nsrc=IlPT2AEptyoA_yixCFOxXnANedT62v3IEQGG_yRJBXSxokDyqRLhGNVoXzb62Sm5HoCb9jHRsxw1wXSaAT-il17","keyword":"","browser":98,"language":37,"screen_color":33,"screen_size":0,"system":13,"platform":62,"operator":2,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":34021738,"uuid":"b00ae536-4ca5-474b-ba2b-fddd4c0b9674","ip":"1879278394","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/53236367.html","engine":0,"referer":"http:\/\/www.ijntv.cn\/inews\/53236367.html","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":2,"country":1,"province":16,"city":0,"flash":"0","java":"0","request_time":1539532800,"create_date":"2018-10-15 00:00:00"}

{"uid":50017547,"uuid":"a76935f7-1da9-4dca-b95f-ba0f298ff5f5","ip":"1972898818","site":0,"source":0,"address":"http:\/\/www.dongfeng.net\/make\/6QRkYJX037Pkyw1co6-DPw.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":2,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539532801,"create_date":"2018-10-15 00:00:01"}

{"uid":50010757,"uuid":"dda000f3-930d-4e2a-8834-7edd05230574","ip":"1018017398","site":0,"source":0,"address":"http:\/\/e.hbqnb.com\/inews\/53383689.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":3,"city":45,"flash":"0","java":"0","request_time":1539532801,"create_date":"2018-10-15 00:00:01"}

{"uid":34004741,"uuid":"ef59dadf-fb58-46fa-aaf1-c1eb8a57e873","ip":"3071113907","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/55373856.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":6,"city":77,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

{"uid":50010336,"uuid":"65b0ac4f-2172-4746-a82e-fc8240240755","ip":"1996541572","site":0,"source":0,"address":"http:\/\/e.hbqnb.com\/inews\/53402956.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":0,"country":1,"province":8,"city":0,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

{"uid":34050322,"uuid":"f9dd0bda-5416-4535-b08b-d91c196977af","ip":"1962504456","site":0,"source":0,"address":"http:\/\/www.ijntv.cn\/inews\/5TJU_3pIEFfcni314q80ZA.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":1,"country":1,"province":30,"city":0,"flash":"0","java":"0","request_time":1539532802,"create_date":"2018-10-15 00:00:02"}

[root@a ~]# tail /data/visitlog/visit-2018-10-15

{"uid":50001890,"uuid":"d1ed8885-280c-4f5d-a06b-112ac2c78b01","ip":"1909901422","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/grFFl7D-XfwzTXUuHIyFiA.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":0,"country":1,"province":0,"city":0,"flash":"0","java":"0","request_time":1539619195,"create_date":"2018-10-15 23:59:55"}

{"uid":50005483,"uuid":"6dcf7b2a-dcd9-460f-8213-0dbe220d474c","ip":"974469123","site":0,"source":0,"address":"http:\/\/www.xichu.net\/look\/45101554.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":3,"country":1,"province":212,"city":0,"flash":"0","java":"0","request_time":1539619196,"create_date":"2018-10-15 23:59:56"}

{"uid":50013267,"uuid":"46245679-8699-48ff-abe2-f691d3f14aa0","ip":"1943015812","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/PMekyfhQyRj8Qzu81ZD6Ww.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":31,"city":384,"flash":"0","java":"0","request_time":1539619197,"create_date":"2018-10-15 23:59:57"}

{"uid":50009806,"uuid":"75d15fb3-8f90-409c-ae16-f6dd58b83979","ip":"3733153632","site":0,"source":0,"address":"http:\/\/www.metalnews.cn\/xinxi\/xlOGEPlObiXPgZGG1ZTFxg.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":0,"country":1,"province":2,"city":0,"flash":"0","java":"0","request_time":1539619197,"create_date":"2018-10-15 23:59:57"}

{"uid":50015377,"uuid":"e70a8aaa-6798-4a49-a3ef-93ff0f6b875c","ip":"2090975143","site":0,"source":0,"address":"http:\/\/www.zjkonline.com\/well\/ycEzO-8D4LBWdiCA6kjb2w.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":3,"country":1,"province":26,"city":342,"flash":"0","java":"0","request_time":1539619198,"create_date":"2018-10-15 23:59:58"}

{"uid":50014416,"uuid":"e7441c1d-f40e-4851-9220-58ed03a416db","ip":"3060659990","site":0,"source":0,"address":"http:\/\/www.zmdtvw.cn\/qyzx\/HYSAdHafvcC5TeNfN_Q9fQ.html","engine":0,"referer":"","keyword":"","browser":11,"language":0,"screen_color":34,"screen_size":0,"system":14,"platform":61,"operator":1,"country":1,"province":17,"city":0,"flash":"0","java":"0","request_time":1539619198,"create_date":"2018-10-15 23:59:58"}

{"uid":50016615,"uuid":"32185523-813f-42b2-aca6-5fb03d0a71a0","ip":"2008640953","site":0,"source":0,"address":"http:\/\/www.zjkonline.com\/well\/QToFaPQVeQ9ix2EuIAp03Q.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":3,"country":1,"province":22,"city":293,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":34032314,"uuid":"315ea4eb-8fa1-4430-a38f-bcbeba450897","ip":"827363088","site":0,"source":0,"address":"http:\/\/www.01hr.com\/bfdx\/45995002.html","engine":0,"referer":"","keyword":"","browser":5,"language":0,"screen_color":34,"screen_size":0,"system":67,"platform":60,"operator":1,"country":1,"province":16,"city":226,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":34032314,"uuid":"315ea4eb-8fa1-4430-a38f-bcbeba450897","ip":"3026902583","site":0,"source":0,"address":"http:\/\/www.01hr.com\/bfdx\/45995002.html","engine":0,"referer":"http:\/\/www.01hr.com\/bfdx\/45995002.html","keyword":"","browser":5,"language":0,"screen_color":34,"screen_size":0,"system":67,"platform":60,"operator":1,"country":1,"province":16,"city":221,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

{"uid":50008572,"uuid":"0fa00371-f97a-429f-bfdd-5bdfdb5b6f92","ip":"3747928108","site":0,"source":0,"address":"http:\/\/www.xxcmw.com\/shangxun\/55410287.html","engine":0,"referer":"","keyword":"","browser":0,"language":0,"screen_color":34,"screen_size":0,"system":0,"platform":0,"operator":2,"country":1,"province":18,"city":0,"flash":"0","java":"0","request_time":1539619199,"create_date":"2018-10-15 23:59:59"}

[root@a ~]#

当前场景借助负载均衡，对网站的访问分发到3个web服务器，同时各自访客日志落入相应web服务器（日志没有做集中化处理）

原方案是借助日志上报，给阿里的日志服务，然后在投入到阿里的dataworks中计算，最后写入mysql

现方案借助开源组件，实现mapReduce过程，得到汇总数据；

同时提高实时性。

Powered by Flink的更多相关文章

腾讯大数据平台Oceanus: A one-stop platform for real time stream processing powered by Apache Flink
January 25, 2019Use Cases, Apache Flink The Big Data Team at Tencent In recent years, the increa ...
Flink应用案例：How Trackunit leverages Flink to process real-time data from industrial IoT devices
January 22, 2019Use Cases, Apache Flink Lasse Nedergaard Recently there has been significant dis ...
Flink写入kafka时，只写入kafka的部分Partitioner，无法写所有的Partitioner问题
1. 写在前面在利用flink实时计算的时候,往往会从kafka读取数据写入数据到kafka,但会发现当kafka多个Partitioner时,特别在P量级数据为了kafka的性能kafka的节点有 ...
从零构建Flink SQL计算平台 - 1平台搭建
一.理想与现实 Apache Flink 是一个分布式流批一体化的开源平台.Flink 的核心是一个提供数据分发.通信以及自动容错的流计算引擎.Flink 在流计算之上构建批处理,并且原生的支持迭代计 ...
apache flink 入门
配置环境包括 JAVA_HOME jobmanager.rpc.address jobmanager.heap.mb 和 taskmanager.heap.mb taskmanager.number ...
Flink 1.1 – ResourceManager
Flink resource manager的作用如图, FlinkResourceManager /** * * <h1>Worker allocation steps</h1 ...
Apache Flink初接触
Apache Flink闻名已久,一直没有亲自尝试一把,这两天看了文档,发现在real-time streaming方面,Flink提供了更多高阶的实用函数. 用Apache Flink实现WordC ...
Flink - InstanceManager
InstanceManager用于管理JobManager申请到的taskManager和slots资源 /** * Simple manager that keeps track of which ...
Flink – window operator
参考, http://wuchong.me/blog/2016/05/25/flink-internals-window-mechanism/ http://wuchong.me/blog/201 ...

随机推荐

[容器]docker创建镜像
手动创建: docker run -d -p mynginx:v2 nginx rpm -ivh http://mirrors.aliyun.com/epel/epel-release-latest- ...
Outlets 和Referencing Outlets的区别
转自:http://blog.sina.com.cn/s/blog_4431c7610100sypy.html 我的理解就是连接与被连接的关系吧 Outlets里面显示的是你的属性, 以及连接着的目标 ...
python学习之items()
定义 Python 字典 items() 方法以列表返回可遍历的(键, 值) 元组数组. 语法 dict.items() 说明:没有参数. 返回值返回可遍历的(键, 值) 元组数组. 例子 > ...
python学习之find()
定义: find() 方法检测字符串中是否包含子字符串 str ,如果指定 beg(开始) 和 end(结束) 范围,则检查是否包含在指定范围内,如果包含子字符串返回开始的索引值,否则返回-1. 语法 ...
河南省第七届ACM程序设计大赛赛后总结
我们学校ACM集训开始于4月5号(清明节),5月25日在郑州解放军信息工程大学举办,集训耗时50天,但是感觉效果还行,但是也不是太好:我们也已经尽力了,虽然说只拿了个银牌,每份收获的背后,都会有辛勤的 ...
Entity简单使用
urlEntity: //定义 package com.example.cc.ecustapp.Model; /** * Created by weijiawang on 2016/3/8. */pu ...
linux学习笔记4--命令mkdir
linux mkdir 命令用来创建指定的名称的目录,要求创建目录的用户在当前目录中具有写权限,并且指定的目录名不能是当前目录中已有的目录. mkdir命令用来创建目录.该命令创建由dirname命名 ...
The Definitive Guide To Django 2 学习笔记(八）第四章模板（四）基本的模板标签和过滤器
标签下面的部分概述了常见的Django标签. if/else {%if%} 标签对一个变量值进行测试,如果结果为true,系统将会显示在{%if%} 和 {%endif%}之间的一切,看个例子: ...
python 脚本撞库国内“某榴”账号
其实日常生活中我们的用户名和密码就那么几个,所以这给撞库带来了可能,本文主要给出python脚本撞库的一点粗浅代码.这里只讨论技术本生,代码中某榴的地址也已经改掉,避免被管理员误解禁言等发生,谢谢大家 ...
解决微信小程序中Date.parse()获取时间戳IOS不兼容的问题(IOS为NaN的问题)
前端同事在做微信小程序时发现IOS获取的时间戳为空的问题,后来通过跟踪发现,原来是因为IOS系统不支持2017-01-01格式的时间导致的, var mydata = '2017-01-01 11:0 ...

Powered by Flink

Powered by Flink

Powered by Flink的更多相关文章

随机推荐

热门专题