最近学习MQTT协议,选择了当前比较流行的MQTT Broker “mosquitto”,但是在阅读代码过程中发现其网络底层库封装的相当差劲。

对于MQTT协议的变长头长度的读取上,基本上采取每次一个byte的方式进行读取判断,对于系统调用read的高代价来讲,真的是相当的浪费,也难怪其不能作为高并发的服务器进行处理。

当然mosquitto需要优化的地方还很多:

1. 使用poll而不是使用epoll (可能是处于跨平台考虑,如果linux下可以使用epoll替换),同时的就是刚才提到的 byte 读取网络数据

2. 订阅树的管理上,对于大量的请求断开或者重练效率比较低

3. 空闲空间管理机制优化和数据包发送方式的修改

4. 内存管理上malloc new 没有使用mem pool机制,在大并发情况下,内存管理容易出现问题

5. 锁遍地飞,如果采用reactor_

但是从另一个方面讲,mosquitto作为开源的实现,思路上还是比较清晰,为mqtt服务器开发提供了比较完备的参考,这也就是它的价值所在了。

#ifdef WITH_BROKER

int _mosquitto_packet_read(struct mosquitto_db *db, struct mosquitto *mosq)

#else

int _mosquitto_packet_read(struct mosquitto *mosq)

#endif

{

uint8_t byte;

ssize_t read_length;

int rc = 0;

if(!mosq) return MOSQ_ERR_INVAL;

if(mosq->sock == INVALID_SOCKET) return MOSQ_ERR_NO_CONN;

if(mosq->state == mosq_cs_connect_pending){

return MOSQ_ERR_SUCCESS;

}

/* This gets called if pselect() indicates that there is network data

* available - ie. at least one byte.  What we do depends on what data we

* already have.

* If we've not got a command, attempt to read one and save it. This should

* always work because it's only a single byte.

* Then try to read the remaining length. This may fail because it is may

* be more than one byte - will need to save data pending next read if it

* does fail.

* Then try to read the remaining payload, where 'payload' here means the

* combined variable header and actual payload. This is the most likely to

* fail due to longer length, so save current data and current position.

* After all data is read, send to _mosquitto_handle_packet() to deal with.

* Finally, free the memory and reset everything to starting conditions.

*/

if(!mosq->in_packet.command){

read_length = _mosquitto_net_read(mosq, &byte, 1);

if(read_length == 1){

mosq->in_packet.command = byte;

#ifdef WITH_BROKER

#  ifdef WITH_SYS_TREE

g_bytes_received++;

#  endif

/* Clients must send CONNECT as their first command. */

if(!(mosq->bridge) && mosq->state == mosq_cs_new && (byte&0xF0) != CONNECT) return MOSQ_ERR_PROTOCOL;

#endif

}else{

if(read_length == 0) return MOSQ_ERR_CONN_LOST; /* EOF */

#ifdef WIN32

errno = WSAGetLastError();

#endif

if(errno == EAGAIN || errno == COMPAT_EWOULDBLOCK){

return MOSQ_ERR_SUCCESS;

}else{

switch(errno){

case COMPAT_ECONNRESET:

return MOSQ_ERR_CONN_LOST;

default:

return MOSQ_ERR_ERRNO;

}

}

}

}

/* remaining_count is the number of bytes that the remaining_length

* parameter occupied in this incoming packet. We don't use it here as such

* (it is used when allocating an outgoing packet), but we must be able to

* determine whether all of the remaining_length parameter has been read.

* remaining_count has three states here:

*   0 means that we haven't read any remaining_length bytes

*   <0 means we have read some remaining_length bytes but haven't finished

*   >0 means we have finished reading the remaining_length bytes.

*/

if(mosq->in_packet.remaining_count <= 0){

do{

read_length = _mosquitto_net_read(mosq, &byte, 1);

if(read_length == 1){

mosq->in_packet.remaining_count--;

/* Max 4 bytes length for remaining length as defined by protocol.

* Anything more likely means a broken/malicious client.

*/

if(mosq->in_packet.remaining_count < -4) return MOSQ_ERR_PROTOCOL;

#if defined(WITH_BROKER) && defined(WITH_SYS_TREE)

g_bytes_received++;

#endif

mosq->in_packet.remaining_length += (byte & 127) * mosq->in_packet.remaining_mult;

mosq->in_packet.remaining_mult *= 128;

}else{

if(read_length == 0) return MOSQ_ERR_CONN_LOST; /* EOF */

#ifdef WIN32

errno = WSAGetLastError();

#endif

if(errno == EAGAIN || errno == COMPAT_EWOULDBLOCK){

return MOSQ_ERR_SUCCESS;

}else{

switch(errno){

case COMPAT_ECONNRESET:

return MOSQ_ERR_CONN_LOST;

default:

return MOSQ_ERR_ERRNO;

}

}

}

}while((byte & 128) != 0);

/* We have finished reading remaining_length, so make remaining_count

* positive. */

mosq->in_packet.remaining_count *= -1;

if(mosq->in_packet.remaining_length > 0){

mosq->in_packet.payload = _mosquitto_malloc(mosq->in_packet.remaining_length*sizeof(uint8_t));

if(!mosq->in_packet.payload) return MOSQ_ERR_NOMEM;

mosq->in_packet.to_process = mosq->in_packet.remaining_length;

}

}

while(mosq->in_packet.to_process>0){

read_length = _mosquitto_net_read(mosq, &(mosq->in_packet.payload[mosq->in_packet.pos]), mosq->in_packet.to_process);

if(read_length > 0){

#if defined(WITH_BROKER) && defined(WITH_SYS_TREE)

g_bytes_received += read_length;

#endif

mosq->in_packet.to_process -= read_length;

mosq->in_packet.pos += read_length;

}else{

#ifdef WIN32

errno = WSAGetLastError();

#endif

if(errno == EAGAIN || errno == COMPAT_EWOULDBLOCK){

if(mosq->in_packet.to_process > 1000){

/* Update last_msg_in time if more than 1000 bytes left to

* receive. Helps when receiving large messages.

* This is an arbitrary limit, but with some consideration.

* If a client can't send 1000 bytes in a second it

* probably shouldn't be using a 1 second keep alive. */

pthread_mutex_lock(&mosq->msgtime_mutex);

mosq->last_msg_in = mosquitto_time();

pthread_mutex_unlock(&mosq->msgtime_mutex);

}

return MOSQ_ERR_SUCCESS;

}else{

switch(errno){

case COMPAT_ECONNRESET:

return MOSQ_ERR_CONN_LOST;

default:

return MOSQ_ERR_ERRNO;

}

}

}

}

/* All data for this packet is read. */

mosq->in_packet.pos = 0;

#ifdef WITH_BROKER

#  ifdef WITH_SYS_TREE

g_msgs_received++;

if(((mosq->in_packet.command)&0xF5) == PUBLISH){

g_pub_msgs_received++;

}

#  endif

rc = mqtt3_packet_handle(db, mosq);

#else

rc = _mosquitto_packet_handle(mosq);

#endif

/* Free data and reset values */

_mosquitto_packet_cleanup(&mosq->in_packet);

pthread_mutex_lock(&mosq->msgtime_mutex);

mosq->last_msg_in = mosquitto_time();

pthread_mutex_unlock(&mosq->msgtime_mutex);

return rc;

}

MQTT 开源代理mosquitto的网络层封装相当sucks的更多相关文章

  1. MQTT开源代理Mosquitto源码分析(访问控制篇)

    一.整体流程概览 从GitHub下载源码后,代理的源码在src中,同时还用到了lib库中的一些函数.对项目的工作流程有个大概理解是分析mosquitto的访问控制权限的基础,网络上已有很多中文博客在介 ...

  2. 【转载】MQTT学习笔记——MQTT协议体验 Mosquitto安装和使用

    http://blog.csdn.net/xukai871105/article/details/39252653 0 前言     MQTT是IBM开发的一个即时通讯协议.MQTT是面向M2M和物联 ...

  3. MQTT入门1 -- mosquitto 安装

    原文链接:https://www.cnblogs.com/NickQ/p/9247638.html MQTT入门1 -- mosquitto 安装 简介: MQTT(Message Queuing T ...

  4. MQTT教學(二):安裝MQTT伺服器Mosquitto,Windows系統篇

    http://swf.com.tw/?p=1005 「認識MQTT」文章提到,MQTT的訊息全都透過稱為代理人(broker)的伺服器交流.本文將說明頗受歡迎的開放原始碼MQTT伺服器Mosquitt ...

  5. 在阿里云服务器上(centos 8) 安装自己的MQTT服务器 (mosquitto)

    layout: post title: 在阿里云服务器上(centos 8) 安装自己的MQTT服务器 (mosquitto) subtitle: date: 2020-3-2 author: Dap ...

  6. DotNetty 版 mqtt 开源客户端 (MqttFx)

    一.DotNetty背景介绍 某天发现 dotnet  是个好东西,就找了个项目来练练手.于是有了本文的 Mqtt 客户端   (github:  MqttFx ) DotNetty是微软的Azure ...

  7. arm linux 移植 MQTT (paho、mosquitto)

    前言 我们在这里做2件事情: 1)编译 paho.mqtt.mosquitto 2个开源项目的c版本库(mosquitto库没有用上) 2)编译好 依赖 paho.mqtt的库编写例程 + mosqu ...

  8. JeeSite是基于多个优秀的开源项目,高度整合封装而成的高效,高性能,强安全性的 开源 Java EE快速开发平台

    JeeSite本身是以Spring Framework为核心容器,Spring MVC为模型视图控制器,MyBatis为数据访问层, Apache Shiro为权限授权层,Ehcahe对常用数据进行缓 ...

  9. 使用Swift的代理,闭包来封装一个公用协议减少垃圾代码

    iOS开发中,如果不进行适当的封装,使用协议或者继承类来进行开发,你就会遇到传说中的ViewController(以后简称VC) Hell的问题…… 比如说,我们先声网App中为了调用接口,做简单的判 ...

随机推荐

  1. Gulp 方法

    Gulp有5个基本方法:src.dest.task.run.watch Gulp.src()      gulp模块的src方法,用于产生数据流.它的参数表示索要处理的文件,一般有以下几种形式: js ...

  2. javascript操作Date对象

    Date 对象用于处理日期和时间. 创建 Date 对象的语法: var myDate=new Date() Date 对象会自动把当前日期和时间保存为其初始值. 参数形式有以下5种: new Dat ...

  3. 用java语言编写的简单二叉树

    package com.cjonline.foundation.evisa; public class TestTree { private int data=-1; private TestTree ...

  4. 分享一个关于pthread线程栈在mm_struct里面的分布问题

    大家好,本人被下面这个问题困扰了一段时间,最近似乎找到了答案. 这里和大家分享一下,可能对有相同困惑的同学有点帮助,同时也请各位帮忙看看错漏的地方. 1================问题: 在使用p ...

  5. 1080Ti+ubuntu14.04

    我来回折腾了几天,从装了好几次系统,后来问了我同学才知道原来是驱动版本的问题,唉,第一次跑去nvidia看他们的online doc.我是相当的郁闷,敢不敢弄得简单点啊,我是电脑小白啊,硬件一窍不通啊 ...

  6. xcode7--iOS开发---将app打包发布至app store

    时隔3个月再次接触应用打包,又是一顿折腾 说说这次的感受吧: 变得是打包时间减少到4小时(其中大部分时间还是xcode7或者是iOS9的原因),不变的是还是一如既往的坑!! 好了,废话不多说,下面讲讲 ...

  7. 2小时学会spring boot 以及spring boot进阶之web进阶(已完成)

    1:更换Maven默认中心仓库的方法 <mirror> <id>nexus-aliyun</id> <mirrorOf>central</mirr ...

  8. jquery mobile 移动web(6)

    jquery mobile 针对移动端设备的事件类型. 1.touch 事件. tap 快速触摸屏幕并且离开,类似一种完整的点击操作. taphold 触摸屏幕并保持一段时间. swipe 在1秒内水 ...

  9. 新疆大学ACM-ICPC程序设计竞赛五月月赛(同步赛)-B-杨老师游戏

    题目链接:杨老师游戏 题目分析:将9个数字分成3块,分块枚举,话句话说,9个数字的所有排列组合,如果满足N=a*b-c就是一个答案,暴力枚举Orz. 代码如下:  #include<iostre ...

  10. poj_3256_Cow Picnic

    The cows are having a picnic! Each of Farmer John's K (1 ≤ K ≤ 100) cows is grazing in one of N (1 ≤ ...