MQTT 开源代理mosquitto的网络层封装相当sucks
最近学习MQTT协议,选择了当前比较流行的MQTT Broker “mosquitto”,但是在阅读代码过程中发现其网络底层库封装的相当差劲。
对于MQTT协议的变长头长度的读取上,基本上采取每次一个byte的方式进行读取判断,对于系统调用read的高代价来讲,真的是相当的浪费,也难怪其不能作为高并发的服务器进行处理。
当然mosquitto需要优化的地方还很多:
1. 使用poll而不是使用epoll (可能是处于跨平台考虑,如果linux下可以使用epoll替换),同时的就是刚才提到的 byte 读取网络数据
2. 订阅树的管理上,对于大量的请求断开或者重练效率比较低
3. 空闲空间管理机制优化和数据包发送方式的修改
4. 内存管理上malloc new 没有使用mem pool机制,在大并发情况下,内存管理容易出现问题
5. 锁遍地飞,如果采用reactor_
但是从另一个方面讲,mosquitto作为开源的实现,思路上还是比较清晰,为mqtt服务器开发提供了比较完备的参考,这也就是它的价值所在了。
#ifdef WITH_BROKER
int _mosquitto_packet_read(struct mosquitto_db *db, struct mosquitto *mosq)
#else
int _mosquitto_packet_read(struct mosquitto *mosq)
#endif
{
uint8_t byte;
ssize_t read_length;
int rc = 0;
if(!mosq) return MOSQ_ERR_INVAL;
if(mosq->sock == INVALID_SOCKET) return MOSQ_ERR_NO_CONN;
if(mosq->state == mosq_cs_connect_pending){
return MOSQ_ERR_SUCCESS;
}
/* This gets called if pselect() indicates that there is network data
* available - ie. at least one byte. What we do depends on what data we
* already have.
* If we've not got a command, attempt to read one and save it. This should
* always work because it's only a single byte.
* Then try to read the remaining length. This may fail because it is may
* be more than one byte - will need to save data pending next read if it
* does fail.
* Then try to read the remaining payload, where 'payload' here means the
* combined variable header and actual payload. This is the most likely to
* fail due to longer length, so save current data and current position.
* After all data is read, send to _mosquitto_handle_packet() to deal with.
* Finally, free the memory and reset everything to starting conditions.
*/
if(!mosq->in_packet.command){
read_length = _mosquitto_net_read(mosq, &byte, 1);
if(read_length == 1){
mosq->in_packet.command = byte;
#ifdef WITH_BROKER
# ifdef WITH_SYS_TREE
g_bytes_received++;
# endif
/* Clients must send CONNECT as their first command. */
if(!(mosq->bridge) && mosq->state == mosq_cs_new && (byte&0xF0) != CONNECT) return MOSQ_ERR_PROTOCOL;
#endif
}else{
if(read_length == 0) return MOSQ_ERR_CONN_LOST; /* EOF */
#ifdef WIN32
errno = WSAGetLastError();
#endif
if(errno == EAGAIN || errno == COMPAT_EWOULDBLOCK){
return MOSQ_ERR_SUCCESS;
}else{
switch(errno){
case COMPAT_ECONNRESET:
return MOSQ_ERR_CONN_LOST;
default:
return MOSQ_ERR_ERRNO;
}
}
}
}
/* remaining_count is the number of bytes that the remaining_length
* parameter occupied in this incoming packet. We don't use it here as such
* (it is used when allocating an outgoing packet), but we must be able to
* determine whether all of the remaining_length parameter has been read.
* remaining_count has three states here:
* 0 means that we haven't read any remaining_length bytes
* <0 means we have read some remaining_length bytes but haven't finished
* >0 means we have finished reading the remaining_length bytes.
*/
if(mosq->in_packet.remaining_count <= 0){
do{
read_length = _mosquitto_net_read(mosq, &byte, 1);
if(read_length == 1){
mosq->in_packet.remaining_count--;
/* Max 4 bytes length for remaining length as defined by protocol.
* Anything more likely means a broken/malicious client.
*/
if(mosq->in_packet.remaining_count < -4) return MOSQ_ERR_PROTOCOL;
#if defined(WITH_BROKER) && defined(WITH_SYS_TREE)
g_bytes_received++;
#endif
mosq->in_packet.remaining_length += (byte & 127) * mosq->in_packet.remaining_mult;
mosq->in_packet.remaining_mult *= 128;
}else{
if(read_length == 0) return MOSQ_ERR_CONN_LOST; /* EOF */
#ifdef WIN32
errno = WSAGetLastError();
#endif
if(errno == EAGAIN || errno == COMPAT_EWOULDBLOCK){
return MOSQ_ERR_SUCCESS;
}else{
switch(errno){
case COMPAT_ECONNRESET:
return MOSQ_ERR_CONN_LOST;
default:
return MOSQ_ERR_ERRNO;
}
}
}
}while((byte & 128) != 0);
/* We have finished reading remaining_length, so make remaining_count
* positive. */
mosq->in_packet.remaining_count *= -1;
if(mosq->in_packet.remaining_length > 0){
mosq->in_packet.payload = _mosquitto_malloc(mosq->in_packet.remaining_length*sizeof(uint8_t));
if(!mosq->in_packet.payload) return MOSQ_ERR_NOMEM;
mosq->in_packet.to_process = mosq->in_packet.remaining_length;
}
}
while(mosq->in_packet.to_process>0){
read_length = _mosquitto_net_read(mosq, &(mosq->in_packet.payload[mosq->in_packet.pos]), mosq->in_packet.to_process);
if(read_length > 0){
#if defined(WITH_BROKER) && defined(WITH_SYS_TREE)
g_bytes_received += read_length;
#endif
mosq->in_packet.to_process -= read_length;
mosq->in_packet.pos += read_length;
}else{
#ifdef WIN32
errno = WSAGetLastError();
#endif
if(errno == EAGAIN || errno == COMPAT_EWOULDBLOCK){
if(mosq->in_packet.to_process > 1000){
/* Update last_msg_in time if more than 1000 bytes left to
* receive. Helps when receiving large messages.
* This is an arbitrary limit, but with some consideration.
* If a client can't send 1000 bytes in a second it
* probably shouldn't be using a 1 second keep alive. */
pthread_mutex_lock(&mosq->msgtime_mutex);
mosq->last_msg_in = mosquitto_time();
pthread_mutex_unlock(&mosq->msgtime_mutex);
}
return MOSQ_ERR_SUCCESS;
}else{
switch(errno){
case COMPAT_ECONNRESET:
return MOSQ_ERR_CONN_LOST;
default:
return MOSQ_ERR_ERRNO;
}
}
}
}
/* All data for this packet is read. */
mosq->in_packet.pos = 0;
#ifdef WITH_BROKER
# ifdef WITH_SYS_TREE
g_msgs_received++;
if(((mosq->in_packet.command)&0xF5) == PUBLISH){
g_pub_msgs_received++;
}
# endif
rc = mqtt3_packet_handle(db, mosq);
#else
rc = _mosquitto_packet_handle(mosq);
#endif
/* Free data and reset values */
_mosquitto_packet_cleanup(&mosq->in_packet);
pthread_mutex_lock(&mosq->msgtime_mutex);
mosq->last_msg_in = mosquitto_time();
pthread_mutex_unlock(&mosq->msgtime_mutex);
return rc;
}
MQTT 开源代理mosquitto的网络层封装相当sucks的更多相关文章
- MQTT开源代理Mosquitto源码分析(访问控制篇)
一.整体流程概览 从GitHub下载源码后,代理的源码在src中,同时还用到了lib库中的一些函数.对项目的工作流程有个大概理解是分析mosquitto的访问控制权限的基础,网络上已有很多中文博客在介 ...
- 【转载】MQTT学习笔记——MQTT协议体验 Mosquitto安装和使用
http://blog.csdn.net/xukai871105/article/details/39252653 0 前言 MQTT是IBM开发的一个即时通讯协议.MQTT是面向M2M和物联 ...
- MQTT入门1 -- mosquitto 安装
原文链接:https://www.cnblogs.com/NickQ/p/9247638.html MQTT入门1 -- mosquitto 安装 简介: MQTT(Message Queuing T ...
- MQTT教學(二):安裝MQTT伺服器Mosquitto,Windows系統篇
http://swf.com.tw/?p=1005 「認識MQTT」文章提到,MQTT的訊息全都透過稱為代理人(broker)的伺服器交流.本文將說明頗受歡迎的開放原始碼MQTT伺服器Mosquitt ...
- 在阿里云服务器上(centos 8) 安装自己的MQTT服务器 (mosquitto)
layout: post title: 在阿里云服务器上(centos 8) 安装自己的MQTT服务器 (mosquitto) subtitle: date: 2020-3-2 author: Dap ...
- DotNetty 版 mqtt 开源客户端 (MqttFx)
一.DotNetty背景介绍 某天发现 dotnet 是个好东西,就找了个项目来练练手.于是有了本文的 Mqtt 客户端 (github: MqttFx ) DotNetty是微软的Azure ...
- arm linux 移植 MQTT (paho、mosquitto)
前言 我们在这里做2件事情: 1)编译 paho.mqtt.mosquitto 2个开源项目的c版本库(mosquitto库没有用上) 2)编译好 依赖 paho.mqtt的库编写例程 + mosqu ...
- JeeSite是基于多个优秀的开源项目,高度整合封装而成的高效,高性能,强安全性的 开源 Java EE快速开发平台
JeeSite本身是以Spring Framework为核心容器,Spring MVC为模型视图控制器,MyBatis为数据访问层, Apache Shiro为权限授权层,Ehcahe对常用数据进行缓 ...
- 使用Swift的代理,闭包来封装一个公用协议减少垃圾代码
iOS开发中,如果不进行适当的封装,使用协议或者继承类来进行开发,你就会遇到传说中的ViewController(以后简称VC) Hell的问题…… 比如说,我们先声网App中为了调用接口,做简单的判 ...
随机推荐
- Gulp 方法
Gulp有5个基本方法:src.dest.task.run.watch Gulp.src() gulp模块的src方法,用于产生数据流.它的参数表示索要处理的文件,一般有以下几种形式: js ...
- javascript操作Date对象
Date 对象用于处理日期和时间. 创建 Date 对象的语法: var myDate=new Date() Date 对象会自动把当前日期和时间保存为其初始值. 参数形式有以下5种: new Dat ...
- 用java语言编写的简单二叉树
package com.cjonline.foundation.evisa; public class TestTree { private int data=-1; private TestTree ...
- 分享一个关于pthread线程栈在mm_struct里面的分布问题
大家好,本人被下面这个问题困扰了一段时间,最近似乎找到了答案. 这里和大家分享一下,可能对有相同困惑的同学有点帮助,同时也请各位帮忙看看错漏的地方. 1================问题: 在使用p ...
- 1080Ti+ubuntu14.04
我来回折腾了几天,从装了好几次系统,后来问了我同学才知道原来是驱动版本的问题,唉,第一次跑去nvidia看他们的online doc.我是相当的郁闷,敢不敢弄得简单点啊,我是电脑小白啊,硬件一窍不通啊 ...
- xcode7--iOS开发---将app打包发布至app store
时隔3个月再次接触应用打包,又是一顿折腾 说说这次的感受吧: 变得是打包时间减少到4小时(其中大部分时间还是xcode7或者是iOS9的原因),不变的是还是一如既往的坑!! 好了,废话不多说,下面讲讲 ...
- 2小时学会spring boot 以及spring boot进阶之web进阶(已完成)
1:更换Maven默认中心仓库的方法 <mirror> <id>nexus-aliyun</id> <mirrorOf>central</mirr ...
- jquery mobile 移动web(6)
jquery mobile 针对移动端设备的事件类型. 1.touch 事件. tap 快速触摸屏幕并且离开,类似一种完整的点击操作. taphold 触摸屏幕并保持一段时间. swipe 在1秒内水 ...
- 新疆大学ACM-ICPC程序设计竞赛五月月赛(同步赛)-B-杨老师游戏
题目链接:杨老师游戏 题目分析:将9个数字分成3块,分块枚举,话句话说,9个数字的所有排列组合,如果满足N=a*b-c就是一个答案,暴力枚举Orz. 代码如下: #include<iostre ...
- poj_3256_Cow Picnic
The cows are having a picnic! Each of Farmer John's K (1 ≤ K ≤ 100) cows is grazing in one of N (1 ≤ ...