Tcpdump MySQL Query
在MySQL线上环境我们一般只打开了binary log,slow log,有时我们需要查看general log呢?因为该log记录所有的请求,打开该日志肯定给磁盘造成很大压力,IO能力有所下降,所以该日志线上一般不打开的,这就到tcpdump闪亮登场了。
tcpdump用法也不算复杂,输出就比较复杂了,如果非常熟悉TCP/IP协议,那么输出对于你来说就是小kiss啦。我们这里只关心MySQL的Query,所以输出还是非常简单,就是日常的查询语句。
1.安装
一般系统都自带了tcpdump命令,没有该命令的采用如下方式安装:
[root@yayun-mysql-server ~]# yum install tcpdump -y
2.简单使用(shell结合perl过滤查询)
[root@yayun-mysql-server ~]# cat query.sh
#!/bin/bash tcpdump -i any -s -l -w - dst port | strings | perl -e '
while(<>) { chomp; next if /^[^ ]+[ ]*$/;
if(/^(SELECT|UPDATE|DELETE|INSERT|SET|COMMIT|ROLLBACK|CREATE|DROP|ALTER|CALL)/i)
{
if (defined $q) { print "$q\n"; }
$q=$_;
} else {
$_ =~ s/^[ \t]+//; $q.=" $_";
}
}'
[root@yayun-mysql-server ~]#
关于各个参数说明请阅读文章最后给的链接。
执行上面脚本,在另外一个窗口执行查询,我使用了sysbench进行压力测试,最后抓取到的结果如下:
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size bytes
{ @ H{ @ H` @ ?h 'f$ ?h ;f$ ?h Of$ ?h cf$ ?h wf$ ?h" f$ ?h# f% ?h# f% H| @ Lg @ ?h+ f% <X _ ?h/ f% ?h/ f% Ha @ #d +/ #d ?/ #d S/ #d g/ #d {/ Hh @ H} @ <e _ L| @ Hb @ ?h? f& ?hM f' ?hN f' ?hZ f( #d3 / H @ Hi @ <x _ <x _ H~ @ Hc @ H} @ S 5 r ,) r @) r T) r h) r v) r! ) ?h~ f+ r+ ) N9 O f+#: f+#: #dM / Lj @ Hd @ #d_ / SHOW TABLE STATUS LIKE 'sbtest'
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
INSERT INTO sbtest values(?,,' ','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
INSERT INTO sbtest values(?,,' ','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
INSERT INTO sbtest values(?,,' ','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
其实还有更简单的方法,那就是使用tcpflow
安装tcpflow
[root@yayun-mysql-server ~]# yum install tcpflow -y
抓取数据的命令如下:
[root@yayun-mysql-server ~]# tcpflow -c -p -i any dst port | grep -i -E "select|insert|update|delete|replace" | sed 's%\(.*\)\([.]\{4\}\)\(.*\)%\3%'
输出结果和tcpdump结果是一样的。
tcpflow[]: listening on any
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
INSERT INTO sbtest values(?,,' ','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
UPDATE sbtest set k=k+ where id=?
UPDATE sbtest set c=? where id=?
DELETE from sbtest where id=?
INSERT INTO sbtest values(?,,' ','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')
SELECT c from sbtest where id=?
SELECT c from sbtest where id between ? and ?
SELECT SUM(K) from sbtest where id between ? and ?
SELECT c from sbtest where id between ? and ? order by c
SELECT DISTINCT c from sbtest where id between ? and ? order by c
最后说说pt-query-digest,这工具包含在percona-toolkit,在分析慢查询方面是非常的好使,具体的用法大家自己前往官网查阅。
我们通过tcpdump抓包以后,通过--type tcpdump选项来分析一下,简单的用法如下:
[root@yayun-mysql-server ~]# tcpdump -s -x -nn -q -tttt -i any -c port > mysql.tcp.txt
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size bytes
packets captured
packets received by filter
packets dropped by kernel
[root@yayun-mysql-server ~]#
然后使用pt-query-digest工具来进行分析一下,用法也比较简单
[root@yayun-mysql-server ~]# pt-query-digest --type tcpdump mysql.tcp.txt # 340ms user time, 50ms system time, 24.38M rss, 205.10M vsz
# Current date: Thu Jun ::
# Hostname: yayun-mysql-server
# Files: mysql.tcp.txt
# Overall: total, unique, 115.61 QPS, .02x concurrency _____________
# Time range: -- ::11.127883 to ::11.300885
# Attribute total min max avg % stddev median
# ============ ======= ======= ======= ======= ======= ======= =======
# Exec time 3ms 51us 862us 171us 568us 191us 84us
# Rows affecte
# Query size
# Warning coun # Profile
# Rank Query ID Response time Calls R/Call V/M Item
# ==== ================== ============= ===== ====== ===== =====
# 0x85FFF5AA78E5FF6A 0.0034 100.0% 0.0002 0.00 BEGIN # Query : 115.61 QPS, .02x concurrency, ID 0x85FFF5AA78E5FF6A at byte
# This item is included in the report because it matches --limit.
# Scores: V/M = 0.00
# Time range: -- ::11.127883 to ::11.300885
# Attribute pct total min max avg % stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count
# Exec time 3ms 51us 862us 171us 568us 191us 84us
# Rows affecte
# Query size
# Warning coun
# String:
# Hosts 192.168.1.20
# Query_time distribution
# 1us
# 10us ################################################################
# 100us ####################################################
# 1ms
# 10ms
# 100ms
# 1s
# 10s+
BEGIN\G
[root@yayun-mysql-server ~]#
如果系统历史比较久,我们想知道哪些账号在连接数据库的话,可以使用下面的命令,非常简单方便哦。
tcpdump -s -x -nn -q -tttt -i any -c port | pt-query-digest --limit=% --type tcpdump | perl -lne 'BEGIN{$/=""};1 while(/Hosts\s+((?:\d{1,3}\.){3}\d{1,3})[\s\S]+?Users\s+(\w+)(?{$h->{"$2\@$1"}++})/mg);END{print $_ for keys %$h}'
参考资料:
http://www.megalinux.net/using-tcpdump-for-mysql-query-logging/
http://www.xfocus.net/articles/200105/172.html
http://www.mysqlperformanceblog.com/2008/11/07/poor-mans-query-logging/
http://www.percona.com/doc/percona-toolkit/2.2/pt-query-digest.html
Tcpdump MySQL Query的更多相关文章
- tcpdump 学习(3):MySQL Query
在MySQL线上环境我们一般只打开了binary log,slow log,有时我们需要查看general log呢?因为该log记录所有的请求,打开该日志肯定给磁盘造成很大压力,IO能力有所下降,所 ...
- Discuz! X3搬家后UCenter出现UCenter info: MySQL Query Error解决方案
Discuz! X3 X2.5论坛搬家后 登录UCenter出现报错:UCenter info: MySQL Query ErrorSQL:SELECT value FROM [Table]vars ...
- MySQL Query Profile
MySQL Query Profiler, 可以查询到此 SQL 语句会执行多少, 并看出 CPU/Memory 使用量, 执行过程 System lock, Table lock 花多少时间等等.从 ...
- Linux下安装MySQL管理工具MySQL Administrator和MySQL Query Browser(转载)
文章来源:http://blog.csdn.net/sunrier/article/details/7572299 Linux下MySQL Administrator和MySQL Query Brow ...
- MySQL Query Cache 相关的问题
最近经常有人问我 MySQL Query Cache 相关的问题,就整理一点 MySQL Query Cache 的内容,以供参考. 顾名思义,MySQL Query Cache 就是用来缓存和 Qu ...
- Mysql query log
一.查询日志的概念: 查询日志记录MySQL中所有的query,通过"--log[=file_name]"来打开该功能.由于记录了所有的query,包括所有的select,体积比较 ...
- MySql Query Cache 优化
query cache原理 当mysql接收到一条select类型的query时,mysql会对这条query进行hash计算而得到一个hash值,然后通过该hash值到query cache中去匹配 ...
- 【原创】3. MYSQL++ Query类型与SQL语句执行过程(非template与SSQLS版本)
我们可以通过使用mysqlpp:: Query来进行SQL语句的增删改查. 首先来看一下mysqlpp::Query的一些最简单的调用, conn.connect(mysqlpp::examples: ...
- mysql query insert中文乱码
mysql新建的表的charset都是utf8的. 在phpmyadmin里直接敲sql,中文可以insert进去的,但是在php代码里mysql_query同样的sql语句就是不行,保存到表里是乱码 ...
随机推荐
- SQL Server -- stuff 函数
STUFF 删除指定长度的字符并在指定的起始点插入另一组字符. 语法 STUFF ( character_expression , start , length , character_express ...
- 如何在Django1.8 结合Python3.4版本中使用MySql
Python2.7时代连接MySql的MySQLdb还不支持Python3.4. pip install pymysql 最关键的一点,在站点的__init__.py文件中,我们添加如下代码: 1 i ...
- day_4_27 py
''' 2018-4-27 19:57:29 其实这些都是讲的类和对象的 self(在定义方法的时候默认的参数)就相当于java里面的this关键字, this.name=name class 类名: ...
- OOM
在mnist上测试卷积网络时,显存不够导致下面的报错: ResourceExhaustedError (see above for traceback): OOM when allocating te ...
- int main(int argc,char *argv[])与int main(int argc,char **argv)区别?
int main(int argc,char *argv[])与int main(int argc,char **argv)区别? 这两种是一个等价的写法 而int main(int argc,cha ...
- [转载]win7x64下的redis安装与使用
原文应用为:`https://www.cnblogs.com/koal/p/5484916.html` 先引用百度百科的一段话吧,具体可以到百科查看吧. Redis是一个开源的使用ANSI C语言编写 ...
- pytorch的torch.utils.data.DataLoader认识
PyTorch中数据读取的一个重要接口是torch.utils.data.DataLoader,该接口定义在dataloader.py脚本中,只要是用PyTorch来训练模型基本都会用到该接口, 该接 ...
- mac休眠掉电快,更改休眠模式
打开终端输入: $ pmset -g 查看休眠模式 hibernatemode 发现值为3, 这是大多数的设置,如果为0 ,那么休眠时严重掉电, 我们可以改变这个模式: $ sudo pmset -a ...
- centos6.5(64bit),python2.6.6安装MySQLdb模块
1.下载MySQL-python.setuptools安装包 2.tar zxvf setuptools-0.6c11.tar.gz 3.cd setuptools-0.6c11 4.python s ...
- 使用XPath对象解析xml文件
使用XPath对象解析xml文件 1.DocumentBuilderFactory类 工厂API,使应用程序能从XML文档获取生成DOM对象树的解析器 其构造方法受保护,用newInstance() ...