mysql压缩表小记
参考文档：
https://www.163.com/dy/article/GI4CH5N305319P76.html
https://learn.lianglianglee.com/专栏/MySQL实战宝典/06 表压缩：不仅仅是空间压缩.md
https://blog.csdn.net/zgaoq/article/details/120522590
https://dev.mysql.com/doc/refman/5.7/en/innodb-page-compression.html
https://dev.mysql.com/doc/refman/5.7/en/innodb-compression-background.html
网上关于这方面资料很多，尤其是姜老师写的最详细，
一、压缩分类
1、COMPRESS 页压缩
2、TPC 压缩
二、自建MySQL环境：
版本：5.7
建表语句：
CREATE TABLE `t1` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` text,
PRIMARY KEY (`id`)
) ENGINE=InnoDB
填充数据：
mysql> insert into t1 select null,repeat('a',200);
Query OK, 1 row affected (0.02 sec)
Records: 1 Duplicates: 0 Warnings: 0
mysql> insert into t1 select null,repeat('a',200) from t1;
Query OK, 1 row affected (0.02 sec)
Records: 1 Duplicates: 0 Warnings: 0
... 重复执行
mysql> insert into t1 select null,repeat('a',200) from t1;
Query OK, 32768 rows affected (0.50 sec)
Records: 32768 Duplicates: 0 Warnings: 0
mysql> select count(*) from t1;
+----------+
| count(*) |
+----------+
| 65536 |
+----------+
1 row in set (0.04 sec)
三、实验 - COMPRESS页压缩
填充的测试数据几乎都是 'a' ，这种数据应该会有很好的压缩性。
COMPRESS 压缩方式主要针对 Innodb 页进行压缩，将一个 16k (innodb 页默认大小)的页面可压缩为8k、4k、2k、1k，如果不指定 key_block_size 大小，该值默认为页的一半大小，也就是8，默认会将一个 16k 的页面压缩为 8k ，这种场景理论上最多将数据压缩为之前的一半。
操作系统查看文件大小
[root@root ceshi]# du -sh *t1*
12K t1.frm
24M t1.ibd
数据库系统表查看文件大小,注意 FILE_SIZE 和 ALLOCATED_SIZE 字段值是一样的
MySQL [ceshi]> select * from information_schema. INNODB_SYS_TABLESPACES where name like '%ceshi/t1%';
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| SPACE | NAME | FLAG | FILE_FORMAT | ROW_FORMAT | PAGE_SIZE | ZIP_PAGE_SIZE | SPACE_TYPE | FS_BLOCK_SIZE | FILE_SIZE | ALLOCATED_SIZE |
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| 832 | ceshi/t1 | 33 | Barracuda | Dynamic | 16384 | 0 | Single | 4096 | 24117248 | 24121344 |
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
1 row in set (0.001 sec)
MySQL [ceshi]> select file_name,total_extents from information_schema.files where file_name like '%ceshi/t1%';
+----------------+---------------+
| file_name | total_extents |
+----------------+---------------+
| ./ceshi/t1.ibd | 23 |
+----------------+---------------+
1 row in set (0.001 sec)
修改 row_format 值，压缩表数据页,注意，默认 key_block_size=8
MySQL [ceshi]> alter table t1 ROW_FORMAT=compressed;
Query OK, 0 rows affected (1.230 sec)
Records: 0 Duplicates: 0 Warnings: 0
MySQL [ceshi]> show create table t1\G
*************************** 1. row ***************************
Table: t1
Create Table: CREATE TABLE `t1` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` text,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=131056 DEFAULT CHARSET=utf8 ROW_FORMAT=COMPRESSED
再次查看文件大小
[root@root ceshi]# du -sh *t1*
12K t1.frm
12M t1.ibd
使用 SQL 查询文件大小
MySQL [ceshi]> select * from information_schema. INNODB_SYS_TABLESPACES where name like '%ceshi/t1%';
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| SPACE | NAME | FLAG | FILE_FORMAT | ROW_FORMAT | PAGE_SIZE | ZIP_PAGE_SIZE | SPACE_TYPE | FS_BLOCK_SIZE | FILE_SIZE | ALLOCATED_SIZE |
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| 833 | ceshi/t1 | 41 | Barracuda | Compressed | 16384 | 8192 | Single | 4096 | 11534336 | 11538432 |
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
1 row in set (0.002 sec)
MySQL [ceshi]> select file_name,total_extents from information_schema.files where file_name like '%ceshi/t1%';
+----------------+---------------+
| file_name | total_extents |
+----------------+---------------+
| ./ceshi/t1.ibd | 11 |
+----------------+---------------+
1 row in set (0.001 sec)
从以上结果看，innodb 文件大小为之前的一半，只有 12MB 了。
修改 key_block_size 值为4、2、1 ，分别查看文件大小
MySQL [ceshi]> alter table t1 KEY_BLOCK_SIZE=4;
Query OK, 0 rows affected (1.464 sec)
Records: 0 Duplicates: 0 Warnings: 0
MySQL [ceshi]> select * from information_schema. INNODB_SYS_TABLESPACES where name like '%ceshi/t1%';
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| SPACE | NAME | FLAG | FILE_FORMAT | ROW_FORMAT | PAGE_SIZE | ZIP_PAGE_SIZE | SPACE_TYPE | FS_BLOCK_SIZE | FILE_SIZE | ALLOCATED_SIZE |
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| 834 | ceshi/t1 | 39 | Barracuda | Compressed | 16384 | 4096 | Single | 4096 | 6291456 | 6295552 |
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
1 row in set (0.001 sec)
MySQL [ceshi]> select file_name,total_extents from information_schema.files where file_name like '%ceshi/t1%';
+----------------+---------------+
| file_name | total_extents |
+----------------+---------------+
| ./ceshi/t1.ibd | 6 |
+----------------+---------------+
1 row in set (0.001 sec)
MySQL [ceshi]> alter table t1 KEY_BLOCK_SIZE=2;
Query OK, 0 rows affected (1.236 sec)
Records: 0 Duplicates: 0 Warnings: 0
MySQL [ceshi]> select * from information_schema. INNODB_SYS_TABLESPACES where name like '%ceshi/t1%';
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| SPACE | NAME | FLAG | FILE_FORMAT | ROW_FORMAT | PAGE_SIZE | ZIP_PAGE_SIZE | SPACE_TYPE | FS_BLOCK_SIZE | FILE_SIZE | ALLOCATED_SIZE |
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| 835 | ceshi/t1 | 37 | Barracuda | Compressed | 16384 | 2048 | Single | 4096 | 3145728 | 3149824 |
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
1 row in set (0.001 sec)
MySQL [ceshi]> select file_name,total_extents from information_schema.files where file_name like '%ceshi/t1%';
+----------------+---------------+
| file_name | total_extents |
+----------------+---------------+
| ./ceshi/t1.ibd | 3 |
+----------------+---------------+
1 row in set (0.001 sec)
MySQL [ceshi]> alter table t1 KEY_BLOCK_SIZE=1;
Query OK, 0 rows affected (2.009 sec)
Records: 0 Duplicates: 0 Warnings: 0
MySQL [ceshi]> select * from information_schema. INNODB_SYS_TABLESPACES where name like '%ceshi/t1%';
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| SPACE | NAME | FLAG | FILE_FORMAT | ROW_FORMAT | PAGE_SIZE | ZIP_PAGE_SIZE | SPACE_TYPE | FS_BLOCK_SIZE | FILE_SIZE | ALLOCATED_SIZE |
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| 836 | ceshi/t1 | 35 | Barracuda | Compressed | 16384 | 1024 | Single | 4096 | 5242880 | 5242880 |
+-------+----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
1 row in set (0.001 sec)
MySQL [ceshi]> select file_name,total_extents from information_schema.files where file_name like '%ceshi/t1%';
+----------------+---------------+
| file_name | total_extents |
+----------------+---------------+
| ./ceshi/t1.ibd | 4 |
+----------------+---------------+
1 row in set (0.002 sec)
小结：
可见并不是 key_block_size 值越小越好。
| key_block_size 值 | 数据文件大小 |
|---|---|
| 不压缩 | 24M |
| 8 | 12M |
| 4 | 6M |
| 2 | 3M |
| 1 | 5M |
三、实验 - TPC表压缩
初始化实验环境，删除重新创建之前的 t1 表；
开始压缩表，并使用 optimize table 命令重建表。
MySQL [ceshi]> alter table t1 COMPRESSION='ZLIB';
Query OK, 0 rows affected (0.011 sec)
Records: 0 Duplicates: 0 Warnings: 0
MySQL [ceshi]> optimize table t1;
+----------+----------+----------+-------------------------------------------------------------------+
| Table | Op | Msg_type | Msg_text |
+----------+----------+----------+-------------------------------------------------------------------+
| ceshi.t1 | optimize | note | Table does not support optimize, doing recreate + analyze instead |
| ceshi.t1 | optimize | status | OK |
+----------+----------+----------+-------------------------------------------------------------------+
2 rows in set (0.604 sec)
操作系统查看文件大小
[root@root ceshi]# du -sh *t1*
12K t1.frm
12M t1.ibd
SQL 查看表大小
MySQL [ceshi]> select * from information_schema. INNODB_SYS_TABLESPACES where name like '%t1%';
+-------+-----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| SPACE | NAME | FLAG | FILE_FORMAT | ROW_FORMAT | PAGE_SIZE | ZIP_PAGE_SIZE | SPACE_TYPE | FS_BLOCK_SIZE | FILE_SIZE | ALLOCATED_SIZE |
+-------+-----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
| 830 | ceshi/t1 | 33 | Barracuda | Dynamic | 16384 | 0 | Single | 4096 | 24117248 | 12066816 |
+-------+-----------+------+-------------+------------+-----------+---------------+------------+---------------+-----------+----------------+
3 rows in set (0.001 sec)
注意看这两个字段，
FILE_SIZE：文件的表面大小，即未压缩文件大小。(ls -l 结果值)
ALLOCATED_SIZE：文件的实际大小，即磁盘上的文件大小。
从结果看，文件缩小了一半，
还有一些压缩相关参数都是默认值，还没时间去做详细测试。
innodb_compression_level
四、实验 - TPC表压缩（aliyun-rds）
参考文章开头，在云上 RDS 初始化环境。
直接看测试结果吧，
mysql> select * from information_schema. INNODB_SYS_TABLESPACES where name like '%t1%';
+-------+------------------------+------+-------------+------------+-----------+---------------+------------+---------------+------------+----------------+
| SPACE | NAME | FLAG | FILE_FORMAT | ROW_FORMAT | PAGE_SIZE | ZIP_PAGE_SIZE | SPACE_TYPE | FS_BLOCK_SIZE | FILE_SIZE | ALLOCATED_SIZE |
+-------+------------------------+------+-------------+------------+-----------+---------------+------------+---------------+------------+----------------+
| 140 | test/t1 | 33 | Barracuda | Dynamic | 16384 | 0 | Single | 4096 | 25165824 | 25120768 |
+-------+------------------------+------+-------------+------------+-----------+---------------+------------+---------------+------------+----------------+
FILE_SIZE 值是24M
开始表级压缩
mysql> alter table t1 COMPRESSION='ZLIB';
Query OK, 0 rows affected (0.02 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> optimize table t1;
+---------+----------+----------+-------------------------------------------------------------------+
| Table | Op | Msg_type | Msg_text |
+---------+----------+----------+-------------------------------------------------------------------+
| test.t1 | optimize | note | Table does not support optimize, doing recreate + analyze instead |
| test.t1 | optimize | status | OK |
+---------+----------+----------+-------------------------------------------------------------------+
2 rows in set (0.37 sec)
mysql> select * from information_schema. INNODB_SYS_TABLESPACES where name like '%t1%';
+-------+------------------------+------+-------------+------------+-----------+---------------+------------+---------------+------------+----------------+
| SPACE | NAME | FLAG | FILE_FORMAT | ROW_FORMAT | PAGE_SIZE | ZIP_PAGE_SIZE | SPACE_TYPE | FS_BLOCK_SIZE | FILE_SIZE | ALLOCATED_SIZE |
+-------+------------------------+------+-------------+------------+-----------+---------------+------------+---------------+------------+----------------+
| 142 | test/t1 | 33 | Barracuda | Dynamic | 16384 | 0 | Single | 4096 | 25165824 | 25120768 |
+-------+------------------------+------+-------------+------------+-----------+---------------+------------+---------------+------------+----------------+
从 FILE_SIZE 结果看，还是24M , 表没有任何变化，如果相同的实验，我放在自建mysql，压缩后大约是12MB。
造成这个的原因也许云上底层文件系统或存储的 block 已经是16kb，这个功能就失去了意义。
mysql压缩表小记的更多相关文章
- mysql压缩表空间
REPAIR TABLE `table_name` 修复表 OPTIMIZE TABLE `table_name` 优化表 OPTIMIZE [LOCAL | NO_WRITE_TO_BINLOG] ...
- MySQL InnoDB表压缩
MySQL InnoDB表压缩 文件大小减小(可达50%以上) ==> 查询速度变快(count * 约减少20%以上时间) 如何设置mysql innodb 表的压缩: 第一,mysql的版本 ...
- 当mysql表从压缩表变成普通表会发生什么
前言 本文章做了把mysql表从压缩表过渡到普通表的实验过程,看看压缩表变成普通表会发生什么?本文针对mysql5.7和mysql8分别进行了实验. 1.什么是表压缩 在介绍压缩表变成普通表前,首先给 ...
- 详解MySQL大表优化方案( 转)
当MySQL单表记录数过大时,增删改查性能都会急剧下降,可以参考以下步骤来优化: 单表优化 除非单表数据未来会一直不断上涨,否则不要一开始就考虑拆分,拆分会带来逻辑.部署.运维的各种复杂度,一般以整型 ...
- MySQL 大表优化方案探讨
当MySQL单表记录数过大时,增删改查性能都会急剧下降,可以参考以下步骤来优化: 单表优化 除非单表数据未来会一直不断上涨,否则不要一开始就考虑拆分,拆分会带来逻辑.部署.运维的各种复杂度,一般以整型 ...
- 谈谈MySQL数据表的类型(转)
谈谈MySQL数据表的类型 通常意义上,数据库也就是数据的集合,具体到计算机上数据库可以是存储器上一些文件的集合或者一些内存数据的集合. 我们通常说的MySql数据库,sql server数据库等等其 ...
- MySQL大表优化方案
转:https://segmentfault.com/a/1190000006158186?hmsr=toutiao.io&utm_medium=toutiao.io&utm_sour ...
- MySQL 大表优化方案
当MySQL单表记录数过大时,增删改查性能都会急剧下降,可以参考以下步骤来优化: 单表优化 除非单表数据未来会一直不断上涨,否则不要一开始就考虑拆分,拆分会带来逻辑.部署.运维的各种复杂度,一般以整型 ...
- 优秀后端架构师必会知识：史上最全MySQL大表优化方案总结
本文原作者“ manong”,原创发表于segmentfault,原文链接:segmentfault.com/a/1190000006158186 1.引言 MySQL作为开源技术的代表作之一,是 ...
- MySQL 大表优化方案（长文）
当MySQL单表记录数过大时,增删改查性能都会急剧下降,可以参考以下步骤来优化: 单表优化 除非单表数据未来会一直不断上涨,否则不要一开始就考虑拆分,拆分会带来逻辑.部署.运维的各种复杂度,一般以整型 ...
随机推荐
- HACKNOS: RECONFORCE (V1.1)
HACKNOS: RECONFORCE (V1.1) 目录 HACKNOS: RECONFORCE (V1.1) 1 信息收集 1.1 端口扫描 1.2 ftp分析 1.3 后台目录扫描 1.2.1 ...
- .Net 6 miniAPI
启动:1.双击 WebApplication1.exe文件 2.dotnet WebApplication1.dll --urls "http://localhost:5403;http ...
- Vue js引用警告 “export ‘default‘ (imported as ‘xxx‘) was not found
问题原因:ES6 编译器识别问题 如果在public.js这样写会有警告export 'default' (imported as 'xxx') was not found export const ...
- PostGIS之空间索引
1. 概述 PostGIS 是PostgreSQL数据库一个空间数据库扩展,它添加了对地理对象的支持,允许在 SQL 中运行空间查询 PostGIS官网:About PostGIS | PostGIS ...
- Apache Maven Assembly自定义打包插件的使用
前言 本文主要记录在SpringBoot项目中使用Apache Maven Assembly插件进行打包的相关内容: 官网说明:https://maven.apache.org/plugins/mav ...
- LeetCode-688 骑士在棋盘上的概率
来源:力扣(LeetCode)链接:https://leetcode-cn.com/problems/knight-probability-in-chessboard 题目描述 在一个 n x n 的 ...
- torch.nn.Embedding使用详解
torch.nn.Embedding: 随机初始化词向量,词向量值在正态分布N(0,1)中随机取值.输入:torch.nn.Embedding(num_embeddings, – 词典的大小尺寸,比如 ...
- pat乙级1022 D进制的A+B
#include<stdio.h> #include<stdlib.h> #include<string.h> #include<math.h> #de ...
- windows11 安装跳过互联网验证
按住shift+F10 cmd 输入 OOBE\BYPASSNRO 电脑重启后 出现 跳过得按钮
- 13.OpenFeign测试远程调用
以会员服务调用优惠券服务为例 引入依赖 在之前创建微服务模块时已经引入了这个依赖,就不需要重复引入了 添加要被member微服务调用的coupon微服务的coupon的方法 在member微服务添加一 ...