在MySQL线上环境我们一般只打开了binary log,slow log,有时我们需要查看general log呢?因为该log记录所有的请求,打开该日志肯定给磁盘造成很大压力,IO能力有所下降,所以该日志线上一般不打开的,这就到tcpdump闪亮登场了。

tcpdump用法也不算复杂,输出就比较复杂了,如果非常熟悉TCP/IP协议,那么输出对于你来说就是小kiss啦。我们这里只关心MySQL的Query,所以输出还是非常简单,就是日常的查询语句。

1.安装

一般系统都自带了tcpdump命令,没有该命令的采用如下方式安装:

[root@yayun-mysql-server ~]# yum install tcpdump -y

2.简单使用(shell结合perl过滤查询)

[root@yayun-mysql-server ~]# cat query.sh
#!/bin/bash tcpdump -i any -s -l -w - dst port | strings | perl -e '
while(<>) { chomp; next if /^[^ ]+[ ]*$/;
if(/^(SELECT|UPDATE|DELETE|INSERT|SET|COMMIT|ROLLBACK|CREATE|DROP|ALTER|CALL)/i)
{
if (defined $q) { print "$q\n"; }
$q=$_;
} else {
$_ =~ s/^[ \t]+//; $q.=" $_";
}
}'
[root@yayun-mysql-server ~]#

关于各个参数说明请阅读文章最后给的链接。
执行上面脚本,在另外一个窗口执行查询,我使用了sysbench进行压力测试,最后抓取到的结果如下:

tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size  bytes
{ @ H{ @ H` @ ?h 'f$ ?h ;f$ ?h Of$ ?h cf$ ?h wf$ ?h" f$ ?h# f% ?h# f% H| @ Lg @ ?h+ f% <X _ ?h/ f% ?h/ f% Ha @ #d +/ #d ?/ #d S/ #d g/ #d {/ Hh @ H} @ <e _ L| @ Hb @ ?h? f& ?hM f' ?hN f' ?hZ f( #d3 / H @ Hi @ <x _ <x _ H~ @ Hc @ H} @ S 5 r ,) r @) r T) r h) r v) r! ) ?h~ f+ r+ ) N9 O f+#: f+#: #dM / Lj @ Hd @ #d_ / SHOW TABLE STATUS LIKE 'sbtest'
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
INSERT INTO sbtest values(?,,' ','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
INSERT INTO sbtest values(?,,' ','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
INSERT INTO sbtest values(?,,' ','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?

其实还有更简单的方法,那就是使用tcpflow

安装tcpflow

[root@yayun-mysql-server ~]# yum install tcpflow -y

抓取数据的命令如下:

[root@yayun-mysql-server ~]# tcpflow -c -p -i any dst port  | grep -i -E "select|insert|update|delete|replace" | sed 's%\(.*\)\([.]\{4\}\)\(.*\)%\3%'  

输出结果和tcpdump结果是一样的。

tcpflow[]: listening on any
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
INSERT INTO sbtest values(?,,' ','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
INSERT INTO sbtest values(?,,' ','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c

最后说说pt-query-digest,这工具包含在percona-toolkit,在分析慢查询方面是非常的好使,具体的用法大家自己前往官网查阅。

我们通过tcpdump抓包以后,通过--type tcpdump选项来分析一下,简单的用法如下:

[root@yayun-mysql-server ~]# tcpdump -s  -x -nn -q -tttt -i any -c  port  > mysql.tcp.txt
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size bytes
packets captured
packets received by filter
packets dropped by kernel
[root@yayun-mysql-server ~]#

然后使用pt-query-digest工具来进行分析一下,用法也比较简单

[root@yayun-mysql-server ~]# pt-query-digest --type tcpdump mysql.tcp.txt                             

# 340ms user time, 50ms system time, 24.38M rss, 205.10M vsz
# Current date: Thu Jun ::
# Hostname: yayun-mysql-server
# Files: mysql.tcp.txt
# Overall: total, unique, 115.61 QPS, .02x concurrency _____________
# Time range: -- ::11.127883 to ::11.300885
# Attribute total min max avg % stddev median
# ============ ======= ======= ======= ======= ======= ======= =======
# Exec time 3ms 51us 862us 171us 568us 191us 84us
# Rows affecte
# Query size
# Warning coun # Profile
# Rank Query ID Response time Calls R/Call V/M Item
# ==== ================== ============= ===== ====== ===== =====
# 0x85FFF5AA78E5FF6A 0.0034 100.0% 0.0002 0.00 BEGIN # Query : 115.61 QPS, .02x concurrency, ID 0x85FFF5AA78E5FF6A at byte
# This item is included in the report because it matches --limit.
# Scores: V/M = 0.00
# Time range: -- ::11.127883 to ::11.300885
# Attribute pct total min max avg % stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count
# Exec time 3ms 51us 862us 171us 568us 191us 84us
# Rows affecte
# Query size
# Warning coun
# String:
# Hosts 192.168.1.20
# Query_time distribution
# 1us
# 10us ################################################################
# 100us ####################################################
# 1ms
# 10ms
# 100ms
# 1s
# 10s+
BEGIN\G
[root@yayun-mysql-server ~]#

如果系统历史比较久,我们想知道哪些账号在连接数据库的话,可以使用下面的命令,非常简单方便哦。

 tcpdump -s  -x -nn -q -tttt -i any -c   port  | pt-query-digest --limit=% --type tcpdump | perl -lne 'BEGIN{$/=""};1 while(/Hosts\s+((?:\d{1,3}\.){3}\d{1,3})[\s\S]+?Users\s+(\w+)(?{$h->{"$2\@$1"}++})/mg);END{print $_ for keys %$h}'

参考资料:

http://www.megalinux.net/using-tcpdump-for-mysql-query-logging/

http://www.xfocus.net/articles/200105/172.html

http://www.mysqlperformanceblog.com/2008/11/07/poor-mans-query-logging/

http://www.percona.com/doc/percona-toolkit/2.2/pt-query-digest.html

Tcpdump MySQL Query的更多相关文章

  1. tcpdump 学习(3):MySQL Query

    在MySQL线上环境我们一般只打开了binary log,slow log,有时我们需要查看general log呢?因为该log记录所有的请求,打开该日志肯定给磁盘造成很大压力,IO能力有所下降,所 ...

  2. Discuz! X3搬家后UCenter出现UCenter info: MySQL Query Error解决方案

    Discuz! X3 X2.5论坛搬家后 登录UCenter出现报错:UCenter info: MySQL Query ErrorSQL:SELECT value FROM [Table]vars ...

  3. MySQL Query Profile

    MySQL Query Profiler, 可以查询到此 SQL 语句会执行多少, 并看出 CPU/Memory 使用量, 执行过程 System lock, Table lock 花多少时间等等.从 ...

  4. Linux下安装MySQL管理工具MySQL Administrator和MySQL Query Browser(转载)

    文章来源:http://blog.csdn.net/sunrier/article/details/7572299 Linux下MySQL Administrator和MySQL Query Brow ...

  5. MySQL Query Cache 相关的问题

    最近经常有人问我 MySQL Query Cache 相关的问题,就整理一点 MySQL Query Cache 的内容,以供参考. 顾名思义,MySQL Query Cache 就是用来缓存和 Qu ...

  6. Mysql query log

    一.查询日志的概念: 查询日志记录MySQL中所有的query,通过"--log[=file_name]"来打开该功能.由于记录了所有的query,包括所有的select,体积比较 ...

  7. MySql Query Cache 优化

    query cache原理 当mysql接收到一条select类型的query时,mysql会对这条query进行hash计算而得到一个hash值,然后通过该hash值到query cache中去匹配 ...

  8. 【原创】3. MYSQL++ Query类型与SQL语句执行过程(非template与SSQLS版本)

    我们可以通过使用mysqlpp:: Query来进行SQL语句的增删改查. 首先来看一下mysqlpp::Query的一些最简单的调用, conn.connect(mysqlpp::examples: ...

  9. mysql query insert中文乱码

    mysql新建的表的charset都是utf8的. 在phpmyadmin里直接敲sql,中文可以insert进去的,但是在php代码里mysql_query同样的sql语句就是不行,保存到表里是乱码 ...

随机推荐

  1. SQL Server -- stuff 函数

    STUFF 删除指定长度的字符并在指定的起始点插入另一组字符. 语法 STUFF ( character_expression , start , length , character_express ...

  2. 如何在Django1.8 结合Python3.4版本中使用MySql

    Python2.7时代连接MySql的MySQLdb还不支持Python3.4. pip install pymysql 最关键的一点,在站点的__init__.py文件中,我们添加如下代码: 1 i ...

  3. day_4_27 py

    ''' 2018-4-27 19:57:29 其实这些都是讲的类和对象的 self(在定义方法的时候默认的参数)就相当于java里面的this关键字, this.name=name class 类名: ...

  4. OOM

    在mnist上测试卷积网络时,显存不够导致下面的报错: ResourceExhaustedError (see above for traceback): OOM when allocating te ...

  5. int main(int argc,char *argv[])与int main(int argc,char **argv)区别?

    int main(int argc,char *argv[])与int main(int argc,char **argv)区别? 这两种是一个等价的写法 而int main(int argc,cha ...

  6. [转载]win7x64下的redis安装与使用

    原文应用为:`https://www.cnblogs.com/koal/p/5484916.html` 先引用百度百科的一段话吧,具体可以到百科查看吧. Redis是一个开源的使用ANSI C语言编写 ...

  7. pytorch的torch.utils.data.DataLoader认识

    PyTorch中数据读取的一个重要接口是torch.utils.data.DataLoader,该接口定义在dataloader.py脚本中,只要是用PyTorch来训练模型基本都会用到该接口, 该接 ...

  8. mac休眠掉电快,更改休眠模式

    打开终端输入: $ pmset -g 查看休眠模式 hibernatemode 发现值为3, 这是大多数的设置,如果为0 ,那么休眠时严重掉电, 我们可以改变这个模式: $ sudo pmset -a ...

  9. centos6.5(64bit),python2.6.6安装MySQLdb模块

    1.下载MySQL-python.setuptools安装包 2.tar zxvf setuptools-0.6c11.tar.gz 3.cd setuptools-0.6c11 4.python s ...

  10. 使用XPath对象解析xml文件

    使用XPath对象解析xml文件 1.DocumentBuilderFactory类  工厂API,使应用程序能从XML文档获取生成DOM对象树的解析器 其构造方法受保护,用newInstance() ...