a bitwise operation 广告投放监控
将随着时间不断增大的数字N个依次编号为1到N的N个球,颜色每次随机为红黑蓝,时间上先后逐个放入篮子中,
计算离现在最近的24个球的红、黑、蓝颜色数
广告投放监控
a bitwise operation
http://redis.io/commands/bitop
http://blog.getspool.com/2011/11/29/fast-easy-realtime-metrics-using-redis-bitmaps/
BITOP operation destkey key [key ...]
Available since 2.6.0.
Time complexity: O(N)
Perform a bitwise operation between multiple keys (containing string values) and store the result in the destination key.
The BITOP command supports four bitwise operations: AND, OR, XOR and NOT, thus the valid forms to call the command are:
BITOP AND destkey srckey1 srckey2 srckey3 ... srckeyN
BITOP OR destkey srckey1 srckey2 srckey3 ... srckeyN
BITOP XOR destkey srckey1 srckey2 srckey3 ... srckeyN
BITOP NOT destkey srckey
As you can see NOT is special as it only takes an input key, because it performs inversion of bits so it only makes sense as an unary operator.
The result of the operation is always stored at destkey
.
Handling of strings with different lengths
When an operation is performed between strings having different lengths, all the strings shorter than the longest string in the set are treated as if they were zero-padded up to the length of the longest string.
The same holds true for non-existent keys, that are considered as a stream of zero bytes up to the length of the longest string.
Return value
The size of the string stored in the destination key, that is equal to the size of the longest input string.
Redis bitmaps – Fast, easy, realtime metrics
At Spool, we calculate our key metrics in real time. Traditionally, metrics are performed by a batch job (running hourly, daily, etc.). Redis backed bitmaps allow us to perform such calculations in realtime and are extremely space efficient. In a simulation of 128 million users, a typical metric such as “daily unique users” takes less than 50 ms on a MacBook Pro and only takes 16 MB of memory. Spool doesn’t have 128 million users yet but it’s nice to know our approach will scale. We thought we’d share how we do it, in case other startups find our approach useful.
Crash Course on Bitmap and Redis Bitmaps
Bitmap (aka Bitset)
A Bitmap or bitset is an array of zeros and ones. A bit in a bitset can be set to either 0
or 1
, and each position in the array is referred to as an offset. Operations such as logical AND, OR, XOR, etc. and other bitwise operations are fair game for Bitmaps.
Population Count
The population count of a Bitmap is the number of bits set to 1
. There are efficient algorithms for calculating population count. For instance, the population count of a 90% filled bitset containing 1 billion bits took 21.1 ms on a MacBook Pro. There is even a hardware instruction in SSE4 for the population count of an integer.
Bitmaps in Redis
Redis allows binary keys and binary values. Bitmaps are nothing but binary values. The setbit(key, offset, value)
operation, which takes O(1)
time, sets the value of a bit to 0
or 1
at the specified offset for a given key.
A simple example: Daily Active Users
To count unique users that logged in today, we set up a bitmap where each user is identified by an offset value. When a user visits a page or performs an action, which warrants it to be counted, set the bit to 1
at the offset representing user id. The key for the bitmap is a function of the name of the action user performed and the timestamp.
In this simple example, every time a user logs in we perform a redis.setbit(daily_active_users, user_id, 1)
. This flips the appropriate offset in the daily_active_users bitmap to 1. This is an O(1) operation. Doing a population count on this results in 9 unique users that logged in today. The key is daily_active_users
and the value is 1011110100100101
.
Of course, since the daily active users will change every day we need a way to create a new bitmap every day. We do this by simply appending the date to the bitmap key. For example, if we want to calculate the daily unique users who have played at least 1 song in a music app for a given day, we can set the key name to be play:yyyy-mm-dd.
If we want to calculate the number of unique users playing a song each hour, we can name the key name will be play:yyyy-mm-dd-hh
. For the rest of the discussion, we will stick with daily unique users that played a song. To collect daily metrics, we will simple set the user’s bit to 1 in the play:yyyy-mm-dd
key whenever a user plays a song. This is an O(1) operation.
redis.setbit(play:yyyy-mm-dd, user_id, 1)
The unique users that played a song today is the population count of the bitmap stored as the value for the play:yyyy-mm-dd
key.To calculate weekly or monthly metrics, we can simply compute the union of all the daily Bitmaps over the week or the month, and then calculate the population count of the resulting bitmap.
You can also extract more complex metrics very easily. For example, the premium account holders who played a song in November would be:
(play:2011-11-01 ∪ play:2011-11-02 ∪...∪play:2011-11-30) ∩ premium:2011-11
Performance comparison using 128 million users
The table below shows a comparison of daily unique action
calculations calculated over 1 day, 7 days and 30 days for 128 million
users. The 7 and 30 metrics are calculated by combining daily bitmaps.
Period | Time (ms) |
---|---|
Daily | 50.2 |
Weekly | 392.0 |
Monthly | 1624.8 |
Optimizations
In the above example, we can optimize the weekly and monthly
computations by caching the calculated daily, weekly, monthly counts in
Redis.
This is a very flexible approach. An added bonus of caching is that
it allows fast cohort analysis, such as weekly unique users who are also
mobile users — the intersection of a mobile users bitmap with a weekly
active users bitmap. Or, if we want to compute rolling unique users over
the last n
days, having cached daily unique counts makes this easy — simply grab the previous n-1
days from your cache and union it with the real time daily count, which only takes 50ms.
Sample Code
A Java code snippet below computes unique users for a given user action and date.
import redis.clients.jedis.Jedis;
import java.util.BitSet;
...
Jedis redis = new Jedis("localhost");
...
public int uniqueCount(String action, String date) {
String key = action + ":" + date;
BitSet users = BitSet.valueOf(redis.get(key.getBytes()));
return users.cardinality();
}
The code snippet below computes the unique users for a given given user action and a list of dates.
import redis.clients.jedis.Jedis;
import java.util.BitSet;
...
Jedis redis = new Jedis("localhost");
...
public int uniqueCount(String action, String... dates) {
BitSet all = new BitSet();
for (String date : dates) {
String key = action + ":" + date;
BitSet users = BitSet.valueOf(redis.get(key.getBytes()));
all.or(users);
}
return all.cardinality();
}
a bitwise operation 广告投放监控的更多相关文章
- 移动互联网广告 - 第十更 - 广告投放运营 DashBoard - 2016/12/10
广告投放运营 DashBoard设计 移动互联网互联网广告投放,数据监控DashBoard,基础样例示意,下图仅供参考(来自于互联网).
- JS魔法堂:再识Bitwise Operation & Bitwise Shift
Brief linkFly的<JavaScript-如果...没有方法>中提及如何手写Math.round方法,各种奇技淫招看着十分过瘾,最让我惊叹的是 ~~(x + )) ,完全通过加法 ...
- 腾讯公司数据分析岗位的hadoop工作 线性回归 k-means算法 朴素贝叶斯算法 SpringMVC组件 某公司的广告投放系统 KNN算法 社交网络模型 SpringMVC注解方式
腾讯公司数据分析岗位的hadoop工作 线性回归 k-means算法 朴素贝叶斯算法 SpringMVC组件 某公司的广告投放系统 KNN算法 社交网络模型 SpringMVC注解方式 某移动公司实时 ...
- RTB--Real TimeBidding模式的互联网广告(实时竞价的广告投放)
RTB(real time bidding)实时竞价允许广告买家根据活动目标.目标人群以及费用门槛等因素对每一个广告及每次广告展示的费用进行竞价.竞价成功后获得广告展示机会,在展示位置上展示广告. 其 ...
- Facebook 广告投放相关概念简介(1)
本文不涉及具体代码实现,仅对开发API的前置内容做简单介绍,想参考代码请绕行! 广告主(广告管理工具) ·需要推广自己的应用.网站.主页,所以有了广告管理工具 . ·一个广告主仅可拥有一个广告账户(可 ...
- 【Tableau】电商广告投放的地域分析
分析师的职责是利用处理数据获取信息,提炼规律,帮助企业正确决策业务方向. 所以,一个好的分析师绝不能被数据所困,既要深入业务,理解业务,也要高瞻远瞩,以领导者的思维借助数据分析的辅助做出判断. [结构 ...
- Android设备广告投放解决方案——大量网络图片、多个网络视频的轮播、缓存与更新
转载请注明原文地址:http://www.cnblogs.com/ygj0930/p/7742996.html 一:业务场景 基于Android系统的设备上投放广告,诸如:地铁广告屏.自助服务机器上的 ...
- HDL之Bitwise operation
1 Verilog 1.1 Bitwise operator Bitwise operators perform a bit wise operation on two operands. They ...
- 智能广告投放平台 All in One
智能广告投放平台 All in One app demos 知之数据 一站式广告营销平台 https://hio.cn/ refs https://www.jonmellman.com/posts/p ...
随机推荐
- 使用 layoutopt 进行布局优化
使用 layoutopt 进行布局优化 Layoutopt 是一款简单的命令行工具,可帮助找到不必要的控件嵌套以及缩减布局资源,从而使应用变得可能“苗条”.控件越少.布局层次越浅,性能就越好. 如果使 ...
- Digital Image Processing 学习笔记2
第二章 2.1视觉感知要素 2.1.1 人眼的结构 眼睛由角膜与巩膜外壳.脉络膜和视网膜包围,晶状体由通信的纤维细胞层组成,并由附在睫状体上的纤维悬挂:视网膜上分布两类光感受器(锥状体和杆状体),他们 ...
- Oracle 11g必须开启的服务及服务详细介绍(转)
成功安装Oracle 11g数据库后,你会发现自己电脑运行速度会变慢,配置较低的电脑甚至出现非常卡的状况,通过禁止非必须开启的Oracle服务可以提升电脑的运行速度.那么,具体该怎么做呢? 按照win ...
- Codeforces Round #313 (Div. 2) A. Currency System in Geraldion
A. Currency System in Geraldion Time Limit: 1 Sec Memory Limit: 256 MB 题目连接 http://codeforces.com/co ...
- zookeeper中client命令实践
Welcome to ZooKeeper! 2016-09-14 16:06:04,528 [myid:] - INFO [main-SendThread(master:2181):ClientCnx ...
- tar.xz文件如何解压 (已验证)
XZ压缩最新压缩率之王 xz这个压缩可能很多都很陌生,不过您可知道xz是绝大数linux默认就带的一个压缩工具. 之前xz使用一直很少,所以几乎没有什么提起. 我是在下载phpmyadmin的时候看到 ...
- SDCard存储
当需要访问SD卡上的文件时,需要按照如下步骤进行 *调用Environment.getExternalStorageState()判读手机上是否插入SD卡(返回MEDIA_MOUNTED则表示已经插入 ...
- 使用jQuery实现类似开关按钮的效果
转自:http://www.cnblogs.com/linjiqin/p/3148228.html 本案例实现类似开关按钮效果. 页面有下拉列表.文本框.按钮等表单元素,大致实现如下效果:1.页面一加 ...
- try-catch 示例
package unit5; import java.util.Scanner; import javax.print.CancelablePrintJob; import javax.sound.m ...
- swift1.2语言函数和闭包函数介绍
swift1.2语言函数和闭包函数介绍 在编程中,随着处理问题的越来越复杂,代码量飞速增加.其中,大量的代码往往相互重复或者近似重复.如果不采有效方式加以解决,代码将很难维护. swift1.2语言函 ...