[Erlang 0129] Erlang 杂记 VI

把之前阅读资料的时候记下的东西,整理了一下.

Adding special-purpose processor support to the Erlang VM

 

P23 简单介绍了Erlang Compiler和Beam文件格式;

The Erlang Compiler in short 章节提到了 Core Erlang 这个之前有提到过:

[Erlang 0120] Know a little Core Erlang

http://www.cnblogs.com/me-sa/p/know_a_little_core_erlang.html

Erlang Beam Format

Python module to parse Erlang BEAM files.

https://github.com/matwey/pybeam

• Erlang BEAM file format

• Erlang external term format

• BEAM file format

P26 3.4 The Erlang Virtual Machine

Joe Armstrong设计的JAM是Stack based machine,现在的BEAM是register based machine.

Beam Loader的章节,解决了之前的很多困惑:

BEAM Loader作为VM的一部分其职责就是加载BEAM文件中的代码.BEAM文件使用的是所谓有限指令集( limited instruction set),这一指令集会在运行时由BEAM Loader展开为扩展指令集(extended instruction set).使用两种指令集的原因是让BEAM文件小一点; BEAM加载逻辑在beam_load.c文件中完成,emulator使用beam_opcodes.c文件中的load scripts将有限指令集转换成为扩展指令集;Perl脚本beam_makeops读取文件ops.tab作为输入生成beam_opcodes.c文件.有限指令集在genop.tab.

上面提到的文件所在路径:

beam_makeops erts/emulator/utils/

ops.tab erts/emulator/beam/

beam_opcodes.c erts/emulator/<machine>/opt/smp/

beam_load.c erts/emulator/beam/

genop.tab lib/compiler/src/

　　

文件beam_opcodes.c是在Erlang编译期生成的.

Exploring Alternative Memory Architectures for Erlang:Implementation and Performance Evaluation

 

地址: http://www.fantasi.se/publications/Wilhelmsson_MSc.pdf

 

这个在之前的学习Erlang垃圾回收机制的时候参考过,这里不再重复:

http://www.cnblogs.com/me-sa/archive/2011/11/13/erlang0014.html

Private Heap 主要的问题在于消息发送都是通过拷贝实现的,这是进程通信的主要消耗,特别是消息体特别大的时候.  http://www.erlang.se/doc/programming_rules.shtml 里面提到:

“Processes are the basic system structuring elements. But do not use processes and message passing when a function call can be used instead.” 

这也就是小消息大运算背后的原因.

@cnDenis   提到的霸爷的文章  量化Erlang进程调度的代价

Heap Architectures for Concurrent Languages using Message Passing

地址: http://www.fantasi.se/publications/ISMM02.pdf

摘要: We describe how interprocess communication and garbage collection happens in each architecture, and extensively discuss the tradeoffs that are involved. In an implementation set-
ting (the Erlang/OTP system) where the rest of the runtime system is unchanged, we present a detailed experimental comparison between these architectures using both synthetic programs and large commercial products as benchmarks.
备注: 一句话总结这篇论文就是:当消息传递的时候本质上发生了什么

P3

Stack用来存放function arguments, return addresses, and local variables.复合数据结构(Compound terms)比如 lists, tuples, 以及超过一个机器字长的浮点数或者bignums都会在heap存放.stack和heap向对方区域增长,这样做的好处是很容易做溢出检测,只要比较一下stack和heap的指针即可,这样设计的缺点是stack或heap的一个内存区域的重新分配会同时影响stack和heap两块区域.Erlang还支持大块的二进制数据,这种数据不是在heap存储而是在单独的全局内存区域存储并进行引用计数.

消息传递是通过拷贝完成的,从sender的heap拷贝到receiver的heap;然后把指向这个消息的指针插入到receiver的mailbox,mailbox包含在进程的PCB内.下面的图

P1进程内有一些共享的数据部分在拷贝的过程中会被展开,这种情况下拷贝之后的消息就要比之前的消息占用更大空间,这种情况实际情况很少发生.这种情况是可以通过跟踪手段(marking technique)和转向指针( forwarding pointers)来规避,但是这样可能让消息通信更慢.

关于forwarding pointer的资料:

http://www.memorymanagement.org/glossary/f.html#glossary-f

forwarding pointer
Some garbage collectors move reachable objects into another space. They leave a forwarding pointer, a special reference pointing to the new location, in the old location.

The following difference between the two memory architectures also deserves to be mentioned: In a process-centric system, it is easy to impose limits on the space resources that a particular (type of) process can use. Doing this in a shared heap system is significantly more complicated and probably quite costly. Currently, this ability is not required by Erlang.

Efficient memory management for concurrent programs that use message passing I,II

地址: http://user.it.uu.se/~kostis/Papers/scp_mm.pdf

备注: 这其实是一个论文集

P13 1.3 Improving message passing
In the private heap architecture, the send operation consists of three parts:
1. Calculate the size of the message to allocate space in the receiver’s heap;
2. copy the message data to the receiver’s heap; and finally,
3. deliver the message to the receiver’s message queue.

To reduce the complexity of the send operation, we want to remove the parts of the send whose cost is proportional to the message size, namely 1 and 2. By introducing a new memory architecture, where all process-local heaps are replaced by a shared memory area, we can achieve this and reduce the cost of sending messages to O(1), that is, make it a constant time operation. We call this the shared heap architecture.

An introduction to Core Erlang
地址: http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=1C8691982F56D28905DAC4D731A386C7?doi=10.1.1.20.7146&rep=rep1&type=pdf

备注: 简单描述Core Erlang 和Erlang的关系,发展历史 从Erlang代码到Core Erlang代码中间经历的分析和转换过程中是怎样被大大简化的.

这个之前有详细了解过, [Erlang 0120] Know a little Core Erlang http://www.cnblogs.com/me-sa/p/know_a_little_core_erlang.html

 

 

All you wanted to know about the HiPE compiler (but might have been afraid to ask)

地址: http://user.it.uu.se/~pergu/papers/erlang03.pdf 

备注:几乎解答了HIPE的所有常见问题

[Erlang 0078] Erlang HiPE 二三事 http://www.cnblogs.com/me-sa/archive/2012/10/09/erlang_hipe.html

 

首先Erlang/OTP compiler完成宏预处理,解析,把一些语法糖还原,之后代码重写成Core Erlang的形式.在Core Erlang环节会完成各种优化比如常量展开(constant folding),函数inline等等.之后代码重写成为BEAM 虚拟机代码,这个环节还有一些优化. 

 

 

1> c(my module, [native]).

compile:file(my module, [native])

erlc +native my module.erl

erlc +native +’{hipe,[verbose]}’ my module.erl

　　

Generating native code and loading it on-the-fly into the system is possible even in cases when the Erlang source code is not available but the .beam file (containing BEAM bytecode) exists. This can be done for whole modules using:

hipe:c(my module)

hipe:c({M,F,A}). 

3> c(my module, [native, {hipe, [verbose, o3]}]).

c(my module, [native, core]).  %% Compilation from Core Erlang

　　

警告:

Do not use the -compile(export_all) directive. This reduces the likelihood of functions being inlined, and makes useful type analysis impossible.