【mysql】关于innodb_file_format
一、几条mysql命令
通过以下命令看一下mysql中 innodb_file_format的配置
mysql> show engines;
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine | Support | Comment | Transactions | XA | Savepoints |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
| CSV | YES | CSV storage engine | NO | NO | NO |
| MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
| BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO |
| MyISAM | YES | MyISAM storage engine | NO | NO | NO |
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
| ARCHIVE | YES | Archive storage engine | NO | NO | NO |
| InnoDB | DEFAULT | Supports transactions, row-level locking, and foreign keys | YES | YES | YES |
| FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL |
+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec) mysql> select version();
+------------+
| version() |
+------------+
| 5.6.25-log |
+------------+
1 row in set (0.00 sec) mysql> select @@innodb_version;
+------------------+
| @@innodb_version |
+------------------+
| 5.6.25 |
+------------------+
1 row in set (0.00 sec) mysql> show variables like "%innodb_file%";
+--------------------------+----------+
| Variable_name | Value |
+--------------------------+----------+
| innodb_file_format | Antelope |
| innodb_file_format_check | ON |
| innodb_file_format_max | Antelope |
| innodb_file_per_table | ON |
+--------------------------+----------+
4 rows in set (0.00 sec) mysql> SET GLOBAL innodb_file_format = barracuda;
Query OK, 0 rows affected (0.00 sec) mysql> show variables like "%innodb_file%";
+--------------------------+-----------+
| Variable_name | Value |
+--------------------------+-----------+
| innodb_file_format | Barracuda |
| innodb_file_format_check | ON |
| innodb_file_format_max | Antelope |
| innodb_file_per_table | ON |
+--------------------------+-----------+
4 rows in set (0.00 sec) mysql> SET GLOBAL innodb_file_format_max = barracuda;
Query OK, 0 rows affected (0.00 sec) mysql> show variables like "%innodb_file%";
+--------------------------+-----------+
| Variable_name | Value |
+--------------------------+-----------+
| innodb_file_format | Barracuda |
| innodb_file_format_check | ON |
| innodb_file_format_max | Barracuda |
| innodb_file_per_table | ON |
+--------------------------+-----------+
4 rows in set (0.01 sec) mysql> show engine innodb status\G
*************************** 1. row ***************************
Type: InnoDB
Name:
Status:
=====================================
2015-12-29 01:31:16 a3731b70 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 11 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 2 srv_active, 0 srv_shutdown, 16015 srv_idle
srv_master_thread log flush and writes: 16017
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 5
OS WAIT ARRAY INFO: signal count 5
Mutex spin waits 0, rounds 0, OS waits 0
RW-shared spins 5, rounds 150, OS waits 5
RW-excl spins 0, rounds 0, OS waits 0
Spin rounds per wait: 0.00 mutex, 30.00 RW-shared, 0.00 RW-excl
------------
TRANSACTIONS
------------
Trx id counter 125966
Purge done for trx's n:o < 124524 undo n:o < 0 state: running but idle
History list length 484
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0, not started
MySQL thread id 41, OS thread handle 0xa3731b70, query id 3846 localhost root init
show engine innodb status
--------
FILE I/O
--------
I/O thread 0 state: waiting for i/o request (insert buffer thread)
I/O thread 1 state: waiting for i/o request (log thread)
I/O thread 2 state: waiting for i/o request (read thread)
I/O thread 3 state: waiting for i/o request (read thread)
I/O thread 4 state: waiting for i/o request (read thread)
I/O thread 5 state: waiting for i/o request (read thread)
I/O thread 6 state: waiting for i/o request (write thread)
I/O thread 7 state: waiting for i/o request (write thread)
I/O thread 8 state: waiting for i/o request (write thread)
I/O thread 9 state: waiting for i/o request (write thread)
Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] ,
ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0
Pending flushes (fsync) log: 0; buffer pool: 0
548 OS file reads, 43 OS file writes, 33 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 1256, seg size 1258, 0 merges
merged operations:
insert 0, delete mark 0, delete 0
discarded operations:
insert 0, delete mark 0, delete 0
Hash table size 553193, node heap has 2 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s
---
LOG
---
Log sequence number 1121969516
Log flushed up to 1121969516
Pages flushed up to 1121969516
Last checkpoint at 1121969516
0 pending log writes, 0 pending chkp writes
18 log i/o's done, 0.00 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 136019968; in additional pool allocated 0
Dictionary memory allocated 47602
Buffer pool size 8191
Free buffers 7823
Database pages 366
Old database pages 0
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 356, created 10, written 38
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 366, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
0 read views open inside InnoDB
Main thread process no. 2255, id 2837244784, state: sleeping
Number of rows inserted 0, updated 0, deleted 0, read 0
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================ 1 row in set (0.01 sec)
二、innodb_file_format
Antelope(羚羊)是Built-in-InnoDB(MySQL内置的InnoDB)支持文件格式的代号,有两种“数据表格式”(row_format):Redundant(冗余)、Compact(紧凑)
Barracuda(梭子鱼)是InnoDB Plugin支持的文件格式,在原来的基础上新增了两种数据表格式的支持:Dynamic 和 Compressed
innodb_file_format在配置文件中指定;row_format则在创建数据表时指定
同时Barracude也支持 old redundant and compact row formats
下面是来自stackexchange上面的回答 和 性能测试
1、Benefits of Barracuda and Compression
Question
I've been reading about MySQL's file formats Antelope and Barracuda a while ago, and I wonder if I could benefit with having Barracuda and Compression.
My server is currently using Antelope, as it is the default of MySQL.
I've had many times issues with memory due to the large database I have. My database is increasing every day.It seems Compression is giving benefits to a few people, like:
http://www.mysqlperformanceblog.com/2008/04/23/real-life-use-case-for-barracuda-innodb-file-format/I understand memory and disk space can be lower, but I'm not sure if I understand this (quoted from the article):
"~5% CPU load according to top (from 80-100% mostly waiting for I/O)
0.01 sec average lookup time by primary key (from 1-20 sec before the conversion)"I thought these two things would NOT improve, because if the data is compressed, the server has to uncompress in order to get the original data again, so doesn't that make sense that CPU usage would increase?
Does that benefit you in read/write intensive applications? Would you recommend me to change to Barracuda and Compression?
Are you aware of any issues of Barracuda?
It seems the answer of the following question points a few issues, but since it's from 2011, I'd say they are fixed by now: http://serverfault.com/questions/258022/mysql-innodb-how-to-switch-to-barracuda-format
Answer
Regarding "Dynamic", the non-compressed Barracuda-only format, very little has changed from compact, mainly on how blobs (and any very dynamic fields) are stored. I have never had any issues with compact vs. dynamic, so I can safely recommend Barracuda's dynamic. Remember that Barracuda also supports old redundant and compact row formats.
The article you are mentioning is probably too old (5.1) and, as Peter Z., Percona's CEO, mentions on the comments it may be a bit misleading. That doesn't mean that compression can't be a huge gain depending on the workloads. However, I would recommend you to try it on versions >= 5.6, as both Facebook and Oracle have done lots of improvements about it.
As more recent reference materials, I would recommend you:
- Getting InnoDB Compression Ready for Facebook Scale
- InnoDB Compression Present and Future
- Benchmarking InnoDB page compression performance
In particular, I like Facebook materials as they are third party (no need for an agenda) and they have one of the largest MySQL deployments in the world. As you can see they have had very successful setups combining SSD technology with compression.
Will it benefit you? That will depend on your workload, working set and setup (IOPS, memory). Depending if you are IO bound, CPU bound or memory bound, compression can affect negatively in some cases, by adding extra CPU, memory requirements (both compressed and uncompressed pages are stored on the InnoDB buffer pool) or generating too many compression failures, augmenting the latency. It also depends on the type of data: compression can help a lot with large text blobs, but it may be useless with already-compressed data.
In my experience, in practice, there are people for which compression was a the holy grail of performance and are very happy with it, but on other cases, we had to revert to uncompressed data as no gain was obtained. While a very heavy writing workload may seem like a bad environment for compression, if in your particular case you are not cpu-bound and memory-bound, but iops-bound it may be none-less helpful.
In a nutshell, it is very difficult to predict results, usually you should setup a test environment for benchmarking and then discover why you get better or worse results (and that way you can play with different block sizes, etc.). Barracuda is completely safe. Compression may or not be for you. And you can always experiment with other compression methods like client-side compression of blobs (for example, if you end up CPU-bound) or other 3rd party engines like TokuDB, in which compression is a big priority, as it is focused in performance for larger datasets than InnoDB can handle.
2、Benchmarking InnoDB page compression performance
配置文件
innodb_file_format=barracuda
innodb_file_per_table = 1
innodb_strict_mode=1 #建议加上
创建数据表
CREATE TABLE name (column1 INT PRIMARY KEY) ENGINE=InnoDB
ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=4;
Compact和redumdant的区别在就是在于首部的存存内容区别。
compact的存储格式为首部为一个非NULL的变长字段长度列表
redundant的存储格式为首部是一个字段长度偏移列表(每个字段占用的字节长度及其相应的位移)。
在Antelope中对于变长字段,低于768字节的,不会进行overflow page存储,某些情况下会减少结果集IO.
Sql代码
mysql> create table compressed1 ( acol int ) ENGINE=INNODB;
Query OK, 0 rows affected (0.22 sec) mysql> show create table compressed1\G
*************************** 1. row ***************************
Table: compressed1
Create Table: CREATE TABLE `compressed1` (
`acol` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8
1 row in set (0.05 sec) mysql> create table compressed2 ( acol int ) ENGINE=INNODB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=4;
Query OK, 0 rows affected, 4 warnings (0.12 sec) mysql> show warnings;
+---------+------+-----------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+-----------------------------------------------------------------------+
| Warning | 1478 | InnoDB: KEY_BLOCK_SIZE requires innodb_file_format > Antelope. |
| Warning | 1478 | InnoDB: ignoring KEY_BLOCK_SIZE=4. |
| Warning | 1478 | InnoDB: ROW_FORMAT=COMPRESSED requires innodb_file_format > Antelope. |
| Warning | 1478 | InnoDB: assuming ROW_FORMAT=COMPACT. |
+---------+------+-----------------------------------------------------------------------+
4 rows in set (0.00 sec) mysql> show create table compressed2\G
*************************** 1. row ***************************
Table: compressed2
Create Table: CREATE TABLE `compressed2` (
`acol` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=4
1 row in set (0.00 sec) mysql> show variables like "%innodb_file%";
+--------------------------+----------+
| Variable_name | Value |
+--------------------------+----------+
| innodb_file_format | Antelope |
| innodb_file_format_check | ON |
| innodb_file_format_max | Antelope |
| innodb_file_per_table | ON |
+--------------------------+----------+ //change mysql> show variables like "%innodb_file%";
+--------------------------+-----------+
| Variable_name | Value |
+--------------------------+-----------+
| innodb_file_format | Barracuda |
| innodb_file_format_check | ON |
| innodb_file_format_max | Barracuda |
| innodb_file_per_table | ON |
+--------------------------+-----------+
4 rows in set (0.00 sec) mysql> create table compressed3 ( acol int ) ENGINE=INNODB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=4;
Query OK, 0 rows affected (0.32 sec) mysql> show create table compressed3 \G
*************************** 1. row ***************************
Table: compressed3
Create Table: CREATE TABLE `compressed3` (
`acol` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=4
1 row in set (0.00 sec) //数据文件 -rw-rw---- 1 mysql mysql 8.4K 12月 29 01:18 compressed1.frm
-rw-rw---- 1 mysql mysql 96K 12月 29 01:18 compressed1.ibd
-rw-rw---- 1 mysql mysql 8.4K 12月 29 01:19 compressed2.frm
-rw-rw---- 1 mysql mysql 96K 12月 29 01:19 compressed2.ibd
-rw-rw---- 1 mysql mysql 8.4K 12月 29 03:03 compressed3.frm
-rw-rw---- 1 mysql mysql 64K 12月 29 03:03 compressed3.ibd //compressed3.ibd 是被压缩过的
使用脚本查看文件信息
[root@localhost innodb]# python py_innodb_page_info.py -v /data/mysql/testdb/compressed3.ibd
page offset 00000000, page type <File Space Header>
page offset 00000000, page type <Freshly Allocated Page>
page offset 00000000, page type <Freshly Allocated Page>
page offset 00000000, page type <Freshly Allocated Page>
Total number of page: 4:
Freshly Allocated Page: 3
File Space Header: 1
[root@localhost innodb]# python py_innodb_page_info.py -v /data/mysql/testdb/compressed2.ibd
page offset 00000000, page type <File Space Header>
page offset 00000001, page type <Insert Buffer Bitmap>
page offset 00000002, page type <File Segment inode>
page offset 00000003, page type <B-tree Node>, page level <0000>
page offset 00000000, page type <Freshly Allocated Page>
page offset 00000000, page type <Freshly Allocated Page>
Total number of page: 6:
Freshly Allocated Page: 2
Insert Buffer Bitmap: 1
File Space Header: 1
B-tree Node: 1
File Segment inode: 1
[root@localhost innodb]# python py_innodb_page_info.py -v /data/mysql/testdb/compressed1.ibd
page offset 00000000, page type <File Space Header>
page offset 00000001, page type <Insert Buffer Bitmap>
page offset 00000002, page type <File Segment inode>
page offset 00000003, page type <B-tree Node>, page level <0000>
page offset 00000000, page type <Freshly Allocated Page>
page offset 00000000, page type <Freshly Allocated Page>
Total number of page: 6:
Freshly Allocated Page: 2
Insert Buffer Bitmap: 1
File Space Header: 1
B-tree Node: 1
File Segment inode: 1 [root@localhost innodb]# od -t x1 -j 54 -N 4 /data/mysql/testdb/compressed1.ibd
0000066 00 00 00 00
0000072
[root@localhost innodb]# od -t x1 -j 54 -N 4 /data/mysql/testdb/compressed2.ibd
0000066 00 00 00 00
0000072
[root@localhost innodb]# od -t x1 -j 54 -N 4 /data/mysql/testdb/compressed3.ibd
0000066 00 00 00 27
0000072
三、文件格式兼容性检查
InnoDB Plugin引入的新的文件格式,也引入较为完整的文件兼容性检查,以防止误操作非兼容的文件格式。兼容性检查一共有三类:启动数据库时、创建数据表时、访问数据表时。
当数据库启动时候,参数innodb_file_format_check(>=5.1.38)会要求InnoDB在启动时检查当前数据表的格式。设置为ON时,如果检测到不支持的格式,那么InnoDB会启动失败;设置为OFF时,检测到不支持的仅会给出警告,并不会导致启动失败。
当创建数据表时,InnoDB会依据参数InnoDB_file_format进行检查,如果创建的数据表格式高于InnoDB_file_format,则创建会失败。
当访问某个数据表(table-access)时,InnoDB也会进行兼容性检查。只要当前运行的InnoDB版本能够支持的格式,都能够被访问,无论参数InnoDB_file_format的配置。
把innodb_file_format_check设置为OFF是很危险的。在InnoDB启动后,一般需要做一些恢复工作,例如Double write buffer/Insert buffer中的数据处理(这依赖于innodb_fast_shutdown参数),试想如果成功启动,但是某些表是不支持的格式,但是InnoDB仍然安装旧版本做恢复,这可能会毁掉相关数据。
所以,一般建议innodb_file_format_check设置为ON。如果是OFF,关闭InnoDB的innodb_fast_shutdown参数务必设置成0
参考文章
http://www.freehao123.com/vps-cpu-io-unixbench/
http://www.orczhou.com/index.php/2010/03/innodb-plugin-file-format/
http://www.orczhou.com/index.php/2014/03/some-tricky-about-mysqladmin-extended-status/
http://www.orczhou.com/index.php/2010/12/more-about-mysql-innodb-shutdown/
http://dba.stackexchange.com/questions/14246/innodb-file-format-barracuda
http://www.tuicool.com/articles/3qm2U3J
http://www.tuicool.com/articles/qQfAJfI
http://www.xuliangwei.com/xubusi/category/%E8%87%AA%E5%8A%A8%E5%8C%96%E8%BF%90%E7%BB%B4
http://www.xuliangwei.com/xubusi/203.html
http://www.xuliangwei.com/xubusi/175.html
http://www.cnblogs.com/billyxp/p/3342969.html?utm_source=tuicool&utm_medium=referral
http://zjadolf.blog.163.com/blog/static/775779202014111063258737/
http://imysql.cn/taxonomy/term/21
http://imysql.com/2015/07/30/mysql-faq-howto-calculate-open-files.shtml
http://imysql.com/2014/09/28/mysql-optimization-case-blob-stored-in-innodb-optimization.shtml
【mysql】关于innodb_file_format的更多相关文章
- MySQL Innodb Engine -- 文件格式(innodb_file_format)
======================================================== 在InnoDB 1.0.x版本之前,InnoDB 存储引擎提供了 Compact 和 ...
- [原] KVM 环境下MySQL性能对比
KVM 环境下MySQL性能对比 标签(空格分隔): Cloud2.0 [TOC] 测试目的 对比MySQL在物理机和KVM环境下性能情况 压测标准 压测遵循单一变量原则,所有的对比都是只改变一个变量 ...
- 如何利用脚本实现MySQL的快速部署以及一机多实例的部署
MySQL有三个版本:二进制,源码包,RPM. 下面讲讲二进制包的安装过程 下载地址: http://dev.mysql.com/downloads/mysql/ 选择Linux-Generic 我这 ...
- [MySQL Reference Manual]14 InnoDB存储引擎
14 InnoDB存储引擎 14 InnoDB存储引擎 14.1 InnoDB说明 14.1.1 InnoDB作为默认存储引擎 14.1.1.1 存储引擎的趋势 14.1.1.2 InnoDB变成默认 ...
- MySQL 5.7 学习:功能性能的提升
背景: 继上次介绍 初识 MySQL 5.6 新功能.参数完之后,刚好MySQL 5.7又GA了,在官方测试里看到,MySQL5.7在功能.性能.可用性.安全和监控上又提升了很高.现在看看和MySQL ...
- mysql 启动不了了
在做测试的工程中,发现mysql启动失败,报错信息如下: --03T08::.969163Z mysqld_safe mysqld from pid file /var/data/my3306/run ...
- MySQL 5.6 & 5.7最优配置模板
摘自:http://mp.weixin.qq.com/s?__biz=MjM5MjIxNDA4NA==&mid=207854835&idx=1&sn=c998031ae6816 ...
- MySQL 升级详细步骤 (包括 Percona)
MySQL 升级步骤 MySQL 5.1.72 升级到 MySQL 5.5.36 鉴于我在升级的时候遇到的麻烦问题,我觉得有必要把一些细节说清楚,免得引起误解了.感觉官方文档上的升级步骤写的比较简单, ...
- 京东MySQL监控之Zabbix优化、自动化
原创作品,允许转载,转载时请务必以超链接形式标明文章 原始出处 .作者信息和本声明.否则将追究法律责任.http://wangwei007.blog.51cto.com/68019/1833332 随 ...
随机推荐
- ios基础之入门(一)
最近找到了一个可以接触ios开发的职位,可以系统的学习和练习了.先从最基本的开始: 一.获取控件的两种方式 1)第一种,也是经常使用的一种,通过IBOutlet方式.直接按住control键,将控件和 ...
- mysqldump命令的常用组合
只导表结构完整语句: mysqldump -h192.168.1.174 --port=3306 -uroot -p --routines --events --no-data --no-cre ...
- ServletFileUpload(Servlet文件上传)
//**文件上传** form表单提交必须指定Enctype 属性指定将数据回发到服务器时浏览器使用的编码类型"multipart/form-data" //1.创建磁盘文件项目工 ...
- 用Qt写软件系列三:一个简单的系统工具之界面美化
前言 在上一篇中,我们基本上完成了主要功能的实现,剩下的一些导出.进程子模块信息等功能,留到后面再来慢慢实现.这一篇来讲述如何对主界面进行个性化的定制.Qt库提供的只是最基本的组件功能,使用这些组件开 ...
- 使用Swift操作NSDate类型基础
时间类型是我们在处理业务的时候使用非常频繁的一个数据类型.下面我们看一下时间NSDate的基本使用方法. 1.比较大小 我比较擅长.NET,我们知道C#里面DateTime类型可以使用"&g ...
- HTML--Table布局
<DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" Content= ...
- 组合数学 + STL --- 利用STL生成全排列
Ignatius and the Princess II Time Limit: 2000/1000 MS (Java/Others) Memory Limit: 65536/32768 K ( ...
- 【WP8】扩展CM的WindowManager
14-09-09更新:修复AppBar冲突bug 关于WindowManager,一直都很想写一篇博客分享一下,一直在忙别的,今天抽空把这个分享一下 在弹窗在移动开发是一个很常见的交互,很多时候我们都 ...
- TCP和UDP之间的区别
TCP---传输控制协议,提供的是面向连接.可靠的字节流服务.当客户和服务器彼此交换数据前,必须先在双方之间建立一个TCP连接,之后才能传输数据.TCP提供超时重发,丢弃重复数据,检验数据,流量控制等 ...
- (转) 关于在IE6下 无法跳转问题
之前在项目,用到超链接,在ie下没有问题,但是到了ie6,居然发现点击事件不起作用, 真不可思议,以前都没注意到,后来网上搜了下,问题就出在这个void(0)上!现把网上的资料整理了下. <a ...