这几天在修改canal, 连接mysql和maria接收到的event有所区别

拿一个简单的insert sql来举例

mysql 会有以下几个event写入到binlog里

1.ANONYMOUS_GTID_LOG_EVENT

2.QUERY_EVENT
header {
  version: 1
  logfileName: "20170105-162017-bin.000001"
  logfileOffset: 1920
  serverId: 1
  serverenCode: "UTF-8"
  executeTime: 1508809530000
  sourceType: MYSQL
  schemaName: ""
  tableName: ""
  eventLength: 72
}
entryType: TRANSACTIONBEGIN
storeValue: " \354\001"

3.TABLE_MAP_EVENT

4.WRITE_ROWS_EVENT
header {
  version: 1
  logfileName: "20170105-162017-bin.000001"
  logfileOffset: 2040
  serverId: 1
  serverenCode: "UTF-8"
  executeTime: 1508809530000
  sourceType: MYSQL
  schemaName: "test"
  tableName: "test1"
  eventLength: 40
  eventType: INSERT
}
entryType: ROWDATA
storeValue: "\b\333\001\020\001P\000b\035\022\033\b\000\020\004\032\002id \000(\0010\000B\00215R\aint(11)"

5.XID_EVENT
header {
  version: 1
  logfileName: "20170105-162017-bin.000001"
  logfileOffset: 2080
  serverId: 1
  serverenCode: "UTF-8"
  executeTime: 1508809530000
  sourceType: MYSQL
  schemaName: ""
  tableName: ""
  eventLength: 31
}
entryType: TRANSACTIONEND
storeValue: "\022\003184"

如果一个事务里不仅一条sql, 4.WRITE_ROWS_EVENT就会有N条, 我们可以注意到一个完整事务TRANSACTIONBEGIN和TRANSACTIONEND必然会配对出现

mysql event部分格式说明

https://dev.mysql.com/doc/internals/en/event-data-for-specific-event-types.html

mariadb 的事件和mysql不一样

1.GTID_EVENT

2.TABLE_MAP_EVENT

3.WRITE_ROWS_EVENT
header {
  version: 1
  logfileName: "20170105-162017-bin.000001"
  logfileOffset: 2040
  serverId: 1
  serverenCode: "UTF-8"
  executeTime: 1508809530000
  sourceType: MYSQL
  schemaName: "test"
  tableName: "test1"
  eventLength: 40
  eventType: INSERT
}
entryType: ROWDATA
storeValue: "\b\333\001\020\001P\000b\035\022\033\b\000\020\004\032\002id \000(\0010\000B\00215R\aint(11)"

4.XID_EVENT
header {
  version: 1
  logfileName: "20170105-162017-bin.000001"
  logfileOffset: 2080
  serverId: 1
  serverenCode: "UTF-8"
  executeTime: 1508809530000
  sourceType: MYSQL
  schemaName: ""
  tableName: ""
  eventLength: 31
}
entryType: TRANSACTIONEND
storeValue: "\022\003184"

gtid_event就是用来替代begin query event的

Binlog event的header格式如下:

4              timestamp
1 event type
4 server-id
4 event-size
4 log pos
2 flags

header的长度固定为19,event type用来标识这个event的类型,event size则是该event包括header的整体长度,而log pos则是下一个event所在的位置

binlog基础格式介绍可参考

http://www.jianshu.com/p/5e6b33d8945f

canal源码里并没有对gtid_event进行解析处理,查看mariadb官网

Event Header

  • Type[1] = 0xa2
  • Flags[2] = 08 00 => LOG_EVENT_SUPPRESS_USE_F

Fields

if flag & FL_GROUP_COMMIT_ID

else

具体地址参考 https://mariadb.com/kb/en/library/gtid_event/

下面是截取到2个事务gtid_event的字节码

41, 0, 0, 34, 0, -17, 0, -71, 75, 51, 90, -94,    2, 0, 0, 0,    38, 0, 0, 0,   56, 20, 0, 0,    8, 0,    115, 25, 0, 0, 0, 0, 0, 0,    0, 0, 0, 0, 12, 0, 0, 0, 0, 0, 0, 31, 0, 0, 0, 0, 0, 0, 0, 11, 0, 0, 0, 0, 3, 8, 8, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 19, 56, 13, 0, 8, 0, 18, 0, 4, 4, 4, 4, 18, 0, 0, -35, 0, 4, 26, 8, 0, 0, 0, 8, 8, 8, 2, 0, 0, 0, 10, 10, 10,

41, 0, 0, 38, 0, -17, 0, -30, 75, 51, 90, -94,    2, 0, 0, 0,    38, 0, 0, 0,   -28, 20,0, 0,    8, 0,    116, 25, 0, 0, 0, 0, 0, 0,    0, 0, 0, 0, 12, 0, 0, 0, 0, 0, 0, 32, 0, 0, 0, 0, 0, 0, 0, 11, 0, 0, 0, 0, 3, 8, 8, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 19, 56, 13, 0, 8, 0, 18, 0, 4, 4, 4, 4, 18, 0, 0, -35, 0, 4, 26, 8, 0, 0, 0, 8, 8, 8, 2, 0, 0, 0, 10, 10, 10, 
红色部分是message header, 41, 0, 0是message长度, 是message sequence_id, -17, 0是semi标识位, -71或者-30开始就是具体event的内容了 115, 25, 0, 0, 0, 0, 0, 0 就是gtid了,用小序列标识=6516
Protocol::Packet

Data between client and server is exchanged in packets of max 16MByte size.
Payload Type Name Description
int<> payload_length Length of the payload. The number of bytes in the packet beyond the initial 4 bytes that make up the packet header.
int<> sequence_id Sequence ID
string<var> payload payload of the packet Example:
01 00 00 00 01 length: 1
sequence_id: x00
payload: 0x01

mysql binlog协议分析--具体event的更多相关文章

  1. mysql 协议分析

    MYSQL Binlog协议分析 此处不讨论建立连接,验证和handshake的交互协议 Binlog协议 一个MYSQL 通信包由包头包体组成 包体根据具体的交互协议有自身的组成结构, 在binlo ...

  2. Mysql Binlog 三种格式介绍及分析

    一.Mysql Binlog格式介绍       Mysql binlog日志有三种格式,分别为Statement,MiXED,以及ROW! 1.Statement:每一条会修改数据的sql都会记录在 ...

  3. Mysql Binlog三种格式介绍及分析【转】

    一.Mysql Binlog格式介绍       Mysql binlog日志有三种格式,分别为Statement,MiXED,以及ROW! 1.Statement:每一条会修改数据的sql都会记录在 ...

  4. Mysql Binlog日志详解

    一.Mysql Binlog格式介绍       Mysql binlog日志有三种格式,分别为Statement,MiXED,以及ROW! 1.Statement:每一条会修改数据的sql都会记录在 ...

  5. Mysql binlog二进制日志

    Mysql binlog日志有三种格式,分别为Statement,MiXED,以及ROW! 1.Statement:每一条会修改数据的实际原sql语句都会被记录在binlog中. 优点:不需要记录每一 ...

  6. 使用python 模仿mybinlog 命令 二进制分析mysql binlog

    出处:黑洞中的奇点 的博客 http://www.cnblogs.com/kelvin19840813/ 您的支持是对博主最大的鼓励,感谢您的认真阅读.本文版权归作者所有,欢迎转载,但请保留该声明. ...

  7. MySQL协议分析

    MySQL协议分析 标签: mysql 2015-02-27 10:22 1807人阅读 评论(1) 收藏 举报  分类: 数据库(19)    目录(?)[+]   1 交互过程 MySQL客户端与 ...

  8. MySQL协议分析2

    MySQL协议分析 议程 协议头 协议类型 网络协议相关函数 NET缓冲 VIO缓冲 MySQL API 协议头 ● 数据变成在网络里传输的数据,需要额外的在头部添加4 个字节的包头. . packe ...

  9. mixer: mysql协议分析

    综述 要实现一个mysql proxy,首先需要做的就是理解并实现mysql通讯协议.这样才能通过proxy架起client到server之间的桥梁. mixer的mysql协议实现主要参考mysql ...

随机推荐

  1. DevExpress v17.2—WPF篇(一)

    用户界面套包DevExpress v17.2终于正式发布,本站将以连载的形式为大家介绍各版本新增内容.本文将介绍了DevExpress WPF v17.2 新的Hamburger Menu.Sched ...

  2. 2019.1.3 WLAN 802.11 a/b/g PHY Specification and EDVT Measurement I - Transmit Power Level

    This lecture provides the WLAN hardware engineer the essential knowledge of IEEE 802.11 a/b/g physic ...

  3. array_merge、array_merge_recursive

    原文:http://www.111cn.net/phper/php/61889.htm    我在php学习在使用到最多的数据合并方法就是array_merge.array_merge_recursi ...

  4. SYSTEM\sys\sys.c:33:7: error: expected '(' before 'void' __ASM void MSR_MSP(u32 addr)

    在STM32中的sys.c文件编译报出这个错误时: __ASM void MSR_MSP(u32 addr){ MSR MSP, r0 //set Main Stack value BX r14} 如 ...

  5. 前端jquery学习--03

    1.tab切换 <!DOCTYPE html> <html lang="en"> <head> <meta charset="U ...

  6. memsql 6.7集群安装

    预备环境处理 安装yum 源 yum install -y yum-utils yum-config-manager --add-repo https://release.memsql.com/pro ...

  7. sql server CLR

    1. 配置sql server 启用CLR 在SQL Server2005/2008里面,CLR默认是关闭的.可以使用如下SQL语句开启CLR. sp_configure 'show advanced ...

  8. 【Android界面实现】AppWidght全面学习之电量监控小部件的实现具体解释

    版权声明:本文为博主原创文章,未经博主同意不得转载. https://blog.csdn.net/bz419927089/article/details/35791047 前几天翻看之前下载的各种资料 ...

  9. C#常用插件和工具

    Code generation(代码自动生成) NVelocity CodeSmith X-Code .NET XGoF - NMatrix / DEVerest Compilation(编译工具) ...

  10. SocketIOCP

    项目地址 :  https://github.com/kelin-xycs/SocketIOCP SocketIOCP 一个 用 C# Socket 实现 的 IOCP 这是一个 用 C# Socke ...