后端程序员之路 36、Apache Kafka

Apache Kafka
http://kafka.apache.org/

Kafka，很容易就联想到《海边的卡夫卡》，文艺程度和Casablanca有得一拼。Kafka是一个分布式消息系统，

Apache Kafka：下一代分布式消息系统
http://www.infoq.com/cn/articles/apache-kafka

基本概念：
    - Topic 主题，表示消息的分类
    - Producer 消息的生存者
    - Consumer 消息的消费者
    - Broker 消息代理

Kafka剖析（一）：Kafka背景及架构介绍
http://www.infoq.com/cn/articles/kafka-analysis-part-1/

c++来使用Kafka最常用的库是librdkafka

edenhill/librdkafka: The Apache Kafka C/C++ library
https://github.com/edenhill/librdkafka

c++作为Consumer的例子：
0、member
    - RdKafka::Consumer* _consumer;
    - RdKafka::Topic* _topic;
1、set kafka compression
    - RdKafka::Conf *conf = RdKafka::Conf::create(RdKafka::Conf::CONF_GLOBAL);
    - string compression = config.get_string(xxx::KAFKA_COMPRESSION);
    - conf->set("compression.codec", compression, errstr)
2、set offset config
    - RdKafka::Conf *tconf = RdKafka::Conf::create(RdKafka::Conf::CONF_TOPIC);
    - tconf->set("auto.commit.enable", ...)、auto.commit.interval.ms、offset.store.method
3、set kafka broker
    - conf->set("metadata.broker.list", broker, errstr)
4、create consumer
    - _consumer = RdKafka::Consumer::create(conf, errstr);
    - _topic = RdKafka::Topic::create(_consumer, topic_str, tconf, errstr);
    - _consumer->start(_topic, _partition, start_offset);
5、consume thread
    - RdKafka::Message *msg = _consumer->consume(_topic, _partition, 1000);
    - _msg_consumer(msg, NULL);
    - delete msg;
    - _consumer->poll(0);