CentOS -- RocketMQ HA & Monitoring

RocketMQ Architecture

NameServer Cluster

Name Servers provide lightweight service discovery and routing. Each Name Server records full routing information, provides corresponding reading and writing service, and supports fast storage expansion, there is no replication among multi-NameServers.

Broker Cluster

Brokers take care of message storage by providing lightweight TOPIC and QUEUE mechanisms. They support the Push and Pull model, contains fault tolerance mechanism (2 copies or 3 copies), and provides strong padding of peaks and capacity of accumulating hundreds of billion messages in their original time order. In addition, Brokers provide disaster recovery, rich metrics statistics, and alert mechanisms, all of which are lacking in traditional messaging systems.

Producer Cluster

Producers support distributed deployment. Distributed Producers send messages to the Broker cluster through multiple load balancing modes. The sending processes support fast failure and have low latency.

Consumer Cluster

Consumers support distributed deployment in the Push and Pull model as well. It also supports cluster consumption and message broadcasting. It provides real-time message subscription mechanism and can meet most consumer requirements. RocketMQ’s website provides a simple quick-start guide to interested users.

HA Deployment Mode

RocketMQ supports various HA options for deployment. Please check out below chart for your reference:

Deployment Mode	Pros	Cons	Comments
Deployment Mode	Pros	Cons	Comments
Single Master	Easy for Deployment and Maintenance	Single point of failure	feasible to Development environments only or for individual usage
Multi-Masters	Easy for configuration and high performance, there is no impact to applications in case of some of masters bouncing or crash.	unconsumed messages on the failed masters cannot be subscribed until the failure got fixed.	Can be used in dev enviorments, better than #1
Multi-Masters/Slaves Async	No impact to applications even if some of masters got crashed. The failover is transparent to aplications	Some messages may got lost after failover due to the async.
Multi-Masters/Slaves Sync	Grantee no messages lost even if some of masters got crashed. The failover is transparent to aplications	Performance isn't as good as #3

Monitoring

• RocketMQ-Console

RocketMQ-Console is not bundled in any release, we can get it from github : apache/rocketmq-externals, it provide comprehensive monitoring information besed on web:

a. Cluster Status

b. Regular maintenance

c. Topic status

d. Producer/Consumers statistics

e. etc...

• Integrated with other monitoring tools with customized scripts, such as Zabbix or Nagios.