Erlang Process input queue
http://www.cnblogs.com/me-sa/archive/2011/11/05/erlang0012.html
Erlang进程有自己的消息队列来保存接收到的消息,新接收到的消息放在队列的尾部。Erlang的接收原语receive就是用来从消息队列中选择性提取消息的。receive提取消息的过程是:从消息队列的第一条消息开始匹配,如果有一条消息可以匹配上就从消息队列中移除,并执行相应的消息处理逻辑。如果没有模式可以匹配消息队列中的消息,这条消息就会保留在消息队列中。如果检查到消息队列中的最后一条消息还没有找到可以匹配的消息,进程就会阻塞直到接收到下一条消息再一次出发提取过程。
我们能不能直观的看到这个过程呢?Erlang对运行时信息的提取提供了很好的支持,我们要查看的是一个进程在运行时的信息,使用的方法:erlang:process_info/1 .这个方法接收的参数就是进程的PID,返回的数据结果:
[{current_function,{looper,loop,0}},
{initial_call,{looper,loop,0}},
{status,waiting},
{message_queue_len,4},
{messages,[abcd,abcde,abcdef,abcdefg]},
{links,[]},{dictionary,[]},
{trap_exit,false},{error_handler,error_handler},
{priority,normal},{group_leader,<0.29.0>},
{total_heap_size,233},{heap_size,233},
{stack_size,1},{reductions,41},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
上面erlang:process_info(Pid).的执行结果中,红色标出就是消息队列中的消息数量,蓝色标出的就是消息队列的内容。下面我们就通过一系列的Demo来理解Erlang进程消息队列的处理机制.
消息队列堆积的情况
我们先把进行测试的脚手架代码准备好,逻辑很简单如果接收到abc消息就输出一下然后继续接收消息,这通过尾递归loop方法就可以实现。
-module(looper).
-compile(export_all).
loop() ->
receive
abc ->
io:format("Receive abc. ~n "),
loop();
stop->
io:format("stop"),
stop
end.
模拟消息堆积的方法很简单,我们不停向这个进程发送无法匹配的消息就可以了,然后我们查看进程的运行时状态,下面是shell中执行的结果,大家看注释:
(demo@192.168.1.123)1> Pid= spawn(looper,loop,[]). %%启动进程返回进程PID
<0.38.0>
(demo@192.168.1.123)2> Pid!abc. %%向进程发送abc消息
Receive abc. %% abc消息正常处理
abc
(demo@192.168.1.123)4> Pid!abcd. %%向进程发送消息abcd
abcd
(demo@192.168.1.123)5> Pid!abcde. %%进程发送消息abcde
abcde
(demo@192.168.1.123)6> Pid!abcdef.
abcdef
(demo@192.168.1.123)7> Pid!abcdefg.
abcdefg
(demo@192.168.1.123)9> erlang:process_info(Pid). %%查看进程状态
[{current_function,{looper,loop,0}},
{initial_call,{looper,loop,0}},
{status,waiting},
{message_queue_len,4},
{messages,[abcd,abcde,abcdef,abcdefg]}, %%这里能看到积压的消息
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,1},
{reductions,41},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)10> Pid!haha. %%再发送一条垃圾消息haha
haha
(demo@192.168.1.123)11> erlang:process_info(Pid).
[{current_function,{looper,loop,0}},
{initial_call,{looper,loop,0}},
{status,waiting},
{message_queue_len,5},
{messages,[abcd,abcde,abcdef,abcdefg,haha]}, %%可以看到haha消息被放在了消息队列的队尾
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,1},
{reductions,41},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)12>
按照优先级接收消息
下面的代码范例来自LYSE,可以看到首先是处理高优先级的消息,如果高优先级的消息处理完毕之后,处理低优先级的消息.
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
important() -> receive {Priority, Message} when Priority > 10 -> [Message | important()] after 0 -> normal() end. normal() -> receive {_, Message} -> [Message | normal()] after 0 -> [] end. |
定时清理堆积的垃圾消息
堆积的垃圾消息会慢慢吃掉内存,而且堆积的消息在Selective Receive过程中会不断地被遍历检查,成为负担,我们现在就添加一个定时清理堆积消息的逻辑:
-module(looper).
-compile(export_all).
%% P= spawn(looper,loop2,[]).
%% erlang:process_info(P).
%%
loop2() ->
receive
abc ->
io:format("Receive abc. ~n "),
loop2();
stop->
io:format("stop"),
stop
after 15000 ->
receive
Any ->
io:format("Receive ~p ~n ",[ Any])
end,
io:format("clear . ~n "),
loop2()
end.
做法也很简单,添加一个超时,超时之后用一个可以接收任意消息(Any)的receive代码段来从消息队列中提取一条消息.为了留出足够的时间来输入命令,我们把超时时间定为15000(15s).好了,启动shell重新来过,大家还是看我添加的注释:
D:\Erlang\Einaction>"C:\Program Files (x86)\erl5.8.2\bin\erl.exe" -name demo@192.168.1.123 -setcookie 123
Eshell V5.8.2 (abort with ^G)
(demo@192.168.1.123)1> P= spawn(looper,loop2,[]). %使用loop2创建进程
<0.38.0>
(demo@192.168.1.123)2> P!abcd. %快速输入下面的命令向进程发送无法匹配的垃圾进程
abcd
(demo@192.168.1.123)3> P!abcde.
abcde
(demo@192.168.1.123)4> P!abcdef.
abcdef
(demo@192.168.1.123)5> P!abcdefg.
abcdefg
(demo@192.168.1.123)6> P!abcdefgg.
abcdefgg
(demo@192.168.1.123)7> erlang:process_info(P).Receive abcd %我们输入完erlang:process_info(P).的时候恰好遇到了清理逻辑执行
(demo@192.168.1.123)7> clear .
(demo@192.168.1.123)7>
[{current_function,{looper,loop2,0}},
{initial_call,{looper,loop2,0}},
{status,waiting},
{message_queue_len,4},
{messages,[abcde,abcdef,abcdefg,abcdefgg]}, %%除了已经被移除掉的abcd 其它垃圾消息还堆积在进程中
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,1},
{reductions,39},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)8> Receive abcde
(demo@192.168.1.123)8> clear .
(demo@192.168.1.123)8> erlang:process_info(P). %上面又执行了一次清理逻辑,我们再次查看进程信息
[{current_function,{looper,loop2,0}},
{initial_call,{looper,loop2,0}},
{status,waiting},
{message_queue_len,3},
{messages,[abcdef,abcdefg,abcdefgg]}, %%看到了吧,又少了一条消息垃圾消息abcde
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,1},
{reductions,69},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)9> Receive abcdef
(demo@192.168.1.123)9> clear .
(demo@192.168.1.123)9> erlang:process_info(P).
[{current_function,{looper,loop2,0}},
{initial_call,{looper,loop2,0}},
{status,waiting},
{message_queue_len,2},
{messages,[abcdefg,abcdefgg]},
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,1},
{reductions,99},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)10> P!abc.
Receive abc.
abc
(demo@192.168.1.123)11> erlang:process_info(P).
[{current_function,{looper,loop2,0}},
{initial_call,{looper,loop2,0}},
{status,waiting},
{message_queue_len,2},
{messages,[abcdefg,abcdefgg]},
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,1},
{reductions,115},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)12> Receive abcdefg
(demo@192.168.1.123)12> clear .
(demo@192.168.1.123)12> Receive abcdefgg
(demo@192.168.1.123)12> clear .
(demo@192.168.1.123)12> erlang:process_info(P).
[{current_function,{looper,loop2,0}},
{initial_call,{looper,loop2,0}},
{status,waiting},
{message_queue_len,0},
{messages,[]}, %好了,执行到这里消息队列已经清空
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,1},
{reductions,175},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)13>
从消息队列中提取消息的慢镜头
receive原语执行过程中,遇到匹配的消息,提取消息进行处理的过程稍纵即逝,我们现在通过添加sleep,来看看这个过程的慢镜头.注意下面代码的修改
-module(looper).
-compile(export_all).
%% P= spawn(looper,loop3,[]).
%% erlang:process_info(P).
%%
loop3() ->
receive
abc ->
io:format("Receive abc. ~n "),
timer:sleep(10000),
io:format("sleep after receive abc done. ~n "),
loop3();
stop->
io:format("stop"),
stop
after 25000 ->
receive
Any ->
io:format("Receive ~p ~n ",[ Any])
end,
io:format("clear . ~n "),
loop3()
end.
下面的shell中,我们向进程发送了一批可以正常处理的abc消息,但是由于处理逻辑中的sleep,消息提取会被拖慢,这个时间我们可以执行process_info
D:\Erlang\Einaction>"C:\Program Files (x86)\erl5.8.2\bin\erl.exe" -name demo@192.168.1.123 -setcookie 123
Eshell V5.8.2 (abort with ^G)
(demo@192.168.1.123)1> P= spawn(looper,loop3,[]).
<0.38.0>
(demo@192.168.1.123)2> erlang:process_info(P).
[{current_function,{looper,loop3,0}},
{initial_call,{looper,loop3,0}},
{status,waiting},
{message_queue_len,0},
{messages,[]},
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,1},
{reductions,9},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)3> P!abc.
Receive abc.
abc
(demo@192.168.1.123)4> P!abc.
abc
(demo@192.168.1.123)5> P!abc.
abc
(demo@192.168.1.123)6> P!abc.
abc
(demo@192.168.1.123)7> P!abc.
abc
(demo@192.168.1.123)8> P!abc.
abc
(demo@192.168.1.123)9> P!abc.
abc
(demo@192.168.1.123)10> P!abc.
abc
(demo@192.168.1.123)11> P!abcd.
abcd
(demo@192.168.1.123)12> P!abcdd.
abcdd
(demo@192.168.1.123)13> erlang:process_info(P).sleep after receive abc done.
(demo@192.168.1.123)13> Receive abc.
(demo@192.168.1.123)13>
[{current_function,{timer,sleep,1}},
{initial_call,{looper,loop3,0}},
{status,waiting},
{message_queue_len,8},
{messages,[abc,abc,abc,abc,abc,abc,abcd,abcdd]},
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,3},
{reductions,66},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)14> erlang:process_info(P).
[{current_function,{timer,sleep,1}},
{initial_call,{looper,loop3,0}},
{status,waiting},
{message_queue_len,8},
{messages,[abc,abc,abc,abc,abc,abc,abcd,abcdd]},
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,3},
{reductions,66},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)15> sleep after receive abc done.
(demo@192.168.1.123)15> Receive abc.
(demo@192.168.1.123)15> sleep after receive abc done.
(demo@192.168.1.123)15> Receive abc.
(demo@192.168.1.123)15> erlang:process_info(P).
[{current_function,{timer,sleep,1}},
{initial_call,{looper,loop3,0}},
{status,waiting},
{message_queue_len,6},
{messages,[abc,abc,abc,abc,abcd,abcdd]},
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,233},
{heap_size,233},
{stack_size,3},
{reductions,131},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,0}]},
{suspending,[]}]
(demo@192.168.1.123)16> sleep after receive abc done.
(demo@192.168.1.123)16> Receive abc.
(demo@192.168.1.123)16> sleep after receive abc done.
(demo@192.168.1.123)16> Receive abc.
(demo@192.168.1.123)16> sleep after receive abc done.
(demo@192.168.1.123)16> Receive abc.
(demo@192.168.1.123)16> sleep after receive abc done.
(demo@192.168.1.123)16> Receive abc.
(demo@192.168.1.123)16> sleep after receive abc done.
(demo@192.168.1.123)16> Receive abcd
(demo@192.168.1.123)16> clear .
(demo@192.168.1.123)16> erlang:process_info(P).
[{current_function,{looper,loop3,0}},
{initial_call,{looper,loop3,0}},
{status,waiting},
{message_queue_len,1},
{messages,[abcdd]}, %执行到这里只有一条垃圾数据堆积了
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,610},
{heap_size,233},
{stack_size,1},
{reductions,305},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,2}]},
{suspending,[]}]
(demo@192.168.1.123)17> Receive abcdd
(demo@192.168.1.123)17> clear .
(demo@192.168.1.123)17> erlang:process_info(P).
[{current_function,{looper,loop3,0}},
{initial_call,{looper,loop3,0}},
{status,waiting},
{message_queue_len,0},
{messages,[]}, %%消息队列已经清空
{links,[]},
{dictionary,[]},
{trap_exit,false},
{error_handler,error_handler},
{priority,normal},
{group_leader,<0.29.0>},
{total_heap_size,610},
{heap_size,233},
{stack_size,1},
{reductions,335},
{garbage_collection,[{min_bin_vheap_size,46368},
{min_heap_size,233},
{fullsweep_after,65535},
{minor_gcs,2}]},
{suspending,[]}]
(demo@192.168.1.123)18>
日出而作子夜还,偶得周末半日闲,各位周末愉快!
2012-4-28 17:45补充
Process Data
For each process there will be at least one =proc_stack and one =proc_heap tag followed by the raw memory information for the stack and heap of the process.
For each process there will also be a =proc_messages tag if the process' message queue is non-empty and a =proc_dictionary tag if the process' dictionary (the put/2 and get/1 thing) is non-empty.The raw memory information can be decoded by the Crashdump Viewer tool. You will then be able to see the stack dump, the message queue (if any) and the dictionary (if any). The stack dump is a dump of the Erlang process stack. Most of the live data (i.e., variables currently in use) are placed on the stack; thus this can be quite interesting. One has to "guess" what's what, but as the information is symbolic, thorough reading of this information can be very useful. As an example we can find the state variable of the Erlang primitive loader on line (5) in the example below:
(1) 3cac44 Return addr 0x13BF58 (<terminate process normally>)
(2) y(0)
["/view/siri_r10_dev/clearcase/otp/erts/lib/kernel/ebin","/view/siri_r10_dev/
(3) clearcase/otp/erts/lib/stdlib/ebin"]
(4) y(1) <0.1.0>
(5) y(2)
{state,[],none,#Fun<erl_prim_loader.6.7085890>,undefined,#Fun<erl_prim_loader.7.900
0327>,#Fun<erl_prim_loader.8.116480692>,#Port<0.2>,infinity,#Fun<erl_prim_loader.9.
10708760>}
(6) y(3) infinity
When interpreting the data for a process, it is helpful to know that anonymous function objects (funs) are given a name constructed from the name of the function in which they are created, and a number (starting with
0) indicating the number of that fun within that function.2012-08-30 16:23 更新
如果需要按照优先级收发消息,可以使用二叉堆(min_heap)或者gb_trees,接收到的消息填充到这样的结构里面(把优先级数值放在第一个key用来排序).使用的时候只需要检索最小或者最大值就可以了.大部分情况下这种方法都可以实现按优先级接受消息,但收到大量高优先级消息的情况就会变慢;从R14A开始,Erlang编译器有一个优化减少了消息接收时进程之间的反复通信,在消息通信之初会创建一个reference附加在往来的消息中,这样在reference创建成功之前自动过滤掉所有不包含这个Reference特征的消息.%% optimized in R14A
optimized(Pid) ->
Ref = make_ref(),
Pid ! {self(), Ref, hello},
receive
{Pid, Ref, Msg} ->
io:format("~p~n", [Msg])
end.
Erlang Process input queue的更多相关文章
- a simple erlang process pool analysis
a simple erlang process pool analysis 这是一个简单的erlang进程池分析,是learn you some erlang for Great Good 里面的一个 ...
- Erlang process structure -- refc binary
Erlang 的process 是虚拟机层面的进程,每个Erlang process 都包括一个 pcb(process control block), 一个stack 以及私有heap . 这部分的 ...
- TF Multi-GPU single input queue
多GPU的数据训练,feed images, labels = cifar10.distorted_inputs() split_images = tf.split(images, FLAGS.num ...
- [Erlang 0057] Erlang 排错利器: Erlang Crash Dump Viewer
http://www.cnblogs.com/me-sa/archive/2012/04/28/2475556.html Erlang Crash Dump Viewer真的是排错的天兵神器,还记得我 ...
- 【原创】大数据基础之Logstash(4)高可用
logstash高可用体现为不丢数据(前提为服务器短时间内不可用后可恢复比如重启服务器或重启进程),具体有两个方面: 进程重启(服务器重启) 事件消息处理失败 在logstash中对应的解决方案为: ...
- 进程 multiprocessing Process join Lock Queue
多道技术 1.空间上的复用 多个程序公用一套计算机硬件 2.时间上的复用 cpu 切换程序+保存程序状态 1.当一个程序遇到IO操作,操作系统会剥夺该程序的cpu执行权限(提高了cpu的利用率,并且不 ...
- Erlang 虚拟机内的内存管理(Lukas Larsson演讲听写稿)
Erlang核心开发者Lukas Larsson在2014年3月份Erlang Factory上的一个演讲详细介绍了Erlang内存体系的原理以及调优案例: http://www.erlang-fac ...
- Erlang使用相关笔记
#从源码编译安装Erlang 1. wget http://www.erlang.org/download/otp_src_r16b.tar.gz -p /usr/local/src 2. tar z ...
- Python之路-python(Queue队列、进程、Gevent协程、Select\Poll\Epoll异步IO与事件驱动)
一.进程: 1.语法 2.进程间通讯 3.进程池 二.Gevent协程 三.Select\Poll\Epoll异步IO与事件驱动 一.进程: 1.语法 简单的启动线程语法 def run(name): ...
随机推荐
- Android .getRGB得到是负数,解决方案
情景:ava.awt.color 下面的getRGB怎么得出的是负数???本来想通过getRGB得到一个整数,在另外的一个部分在根据这个整数构件一个color,因为参数规定只能能传整数!!!color ...
- java线程——详解Callable、Future和FutureTask
回顾: 接上篇博客 java线程--三种创建线程的方式,这篇博客主要介绍第三种方式Callable和Future.比较继承Thread类和实现Runnable接口,接口更加灵活,使用更广泛.但这两种方 ...
- 洛谷 P1192 台阶问题
P1192 台阶问题 题目描述 有N级的台阶,你一开始在底部,每次可以向上迈最多K级台阶(最少1级),问到达第N级台阶有多少种不同方式. 输入输出格式 输入格式: 输入文件的仅包含两个正整数N,K. ...
- numpy 高阶函数 —— np.histogram
np.diff(a, n=1, axis=-1):n 表示差分的阶数: >> x = np.array([1, 2, 4, 7, 0]) >> np.diff(x) array ...
- (转)利用openfiler实现iSCSI
转自:http://czmmiao.iteye.com/blog/1735417 openfiler openfiler是一个基于浏览器的网络存储管理工具.来自于Linux系统.openfiler在一 ...
- 【习题 5-8 UVA - 230】Borrowers
[链接] 我是链接,点我呀:) [题意] 在这里输入题意 [题解] 用map+set写个模拟就好. 3个区域 书架.桌子.别人的手上. 其中前两个区域的书都能借出去. [代码] #include &l ...
- 苹果APNS在app中的详细实现
鉴于server稳定的开发难度非常大,小团队不建议自己开发.建议使用稳定的第三方推送方案,如个推,蝴蝶等. 要想使用苹果APNS推送消息,首先要把开发app的xcode所用证书上传到server上,当 ...
- POJ 1163 The Triangle 简单DP
看题传送门门:http://poj.org/problem?id=1163 困死了....QAQ 普通做法,从下往上,可得状态转移方程为: dp[i][j]= a[i][j] + max (dp[i+ ...
- oracle数据库的备份与恢复
一.备份 方法1: PLSQL中进行导出 对于方式1: 对于导出可执行文件的选择,可通过下面的几个参考位置去查找: 导入imp:F:\app\Administrator\product\11.1 ...
- 对象模型图(OMD)阅读指南
樱木 原文 对象模型图(OMD)阅读指南(转载) 补充几个名词概念: UML:Unified Modeling Language 统一建模语言,是用来对软件密集系统进行可视化建模的一种语言.UML为面 ...
