openstack组件通讯端口定义

openstack 组件通讯是通过ZeroMQ+ceilometer发送组件调用信息，具体是通过TCP通讯，发送数据和接收数据是用同一个端口（在配置文件指定），下面通过代码稍作解析：

IceHouse/ceilometer/ceilometer/openstack/common/rpc/impl_zmq.py

def _multi_send(method, context, topic, msg, timeout=None,
                envelope=False, _msg_id=None):
    """Wraps the sending of messages.

    Dispatches to the matchmaker and sends message to all relevant hosts.
    """
    conf = CONF
    LOG.debug(_("%(msg)s") % {'msg': ' '.join(map(pformat, (topic, msg)))})

    queues = _get_matchmaker().queues(topic)
    LOG.debug(_("Sending message(s) to: %s"), queues)

    # Don't stack if we have no matchmaker results
    if not queues:
        LOG.warn(_("No matchmaker results. Not casting."))
        # While not strictly a timeout, callers know how to handle
        # this exception and a timeout isn't too big a lie.
        raise rpc_common.Timeout(_("No match from matchmaker."))

    # This supports brokerless fanout (addresses > 1)
    for queue in queues:
        (_topic, ip_addr) = queue
        _addr = "tcp://%s:%s" % (ip_addr, conf.rpc_zmq_port)

        if method.__name__ == '_cast':
            eventlet.spawn_n(method, _addr, context,
                             _topic, msg, timeout, envelope,
                             _msg_id)
            return
        return method(_addr, context, _topic, msg, timeout,
                      envelope)

debug日志:/var/log/ceilometer/compute.log

compute.log:-- ::58.972  DEBUG ceilometer.openstack.common.rpc.common [-] Sending message(s) to: [(u'metering.ceilometer', u'ceilometer')] _multi_send /usr/lib/python2./site-packages/ceilometer/openstack/common/rpc/impl_zmq.py:
compute.log:-- ::58.982  DEBUG ceilometer.openstack.common.rpc.common [-] Sending message(s) to: [(u'metering.ceilometer', u'ceilometer')] _multi_send /usr/lib/python2./site-packages/ceilometer/openstack/common/rpc/impl_zmq.py:

1. conf.rpc_zmq_port是通过/etc/nova/nova.conf中指定的，也即是nova服务（/usr/bin/oslo-messaging-zmq-receiver）启动的监听端口：

[root@test1 ceilometer]# netstat -lntp | grep
tcp               192.168.213.202:        0.0.0.0:*                   LISTEN      /haproxy
tcp               192.168.213.88:         0.0.0.0:*                   LISTEN      /python
[root@test1 ceilometer]# ps -ef | grep
nova              May13 ?        :: /usr/bin/python /usr/bin/oslo-messaging-zmq-receiver --config-file /etc/nova/nova.conf
root         : pts/    :: grep 

2. 通过上面代码可以看到nova与ceilometer通讯的指定的目的端口也是rpc_zmq_port，由此可以得出，在all in one环境中，需要对cinder、nova、neutron和ceilometer做端口转发，才能在不修改源码的基础上做到组件之间通讯(组件与ceilometer建立tcp连接)，用haproxy：

/etc/haproxy/haproxy.cfg

#/etc/haproxy/haproxy.cfg

#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         /dev/log local2

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     32768
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    tcp
    log                     global
    #option                  httplog
    option                  dontlognull
    #option http-server-close
    #option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 32768

listen haproxy_stats
    bind *:8888
    mode http
    stats uri /
    #stats refresh 5
    # stats realm HAproxy\ stats
    # stats auth admin:admin

listen zmq-cinder
    bind 192.168.213.202:9501
    mode tcp
    #log         127.0.0.1 local2
    balance roundrobin
    server test1 192.168.213.88:9500

listen  zmq-nova
    bind 192.168.213.202:9502
    mode tcp
    #log         127.0.0.1 local2
    balance roundrobin
    server test1 192.168.213.88:9500

listen  zmq-neutron
    bind 192.168.213.202:9505
    mode tcp
    #log         127.0.0.1 local2
    balance roundrobin
    server test1 192.168.213.88:9500

listen  zmq-ceilometer
    bind 192.168.213.202:9500
    mode tcp
    #log         127.0.0.1 local2
    balance roundrobin
    server test1 192.168.213.88:9500

3. 建立TCP连接时指定目的IP是在nova配置文件的[matchmaker_ring]模块中指定：

[matchmaker_ring]
ringfile = /etc/nova/matchmaker_ring.json

{
    "conductor":["test1"],
    "scheduler":["test1"],
    "compute": ["test1"],
    "cert": ["test1"],
    "consoleauth":["test1"],
    "backendtask":["test1"],
    "metering": ["ceilometer"],
    "notifications-info": ["ceilometer"],
    "notifications-error": ["ceilometer"]
}