[转帖]金仓数据库KingbaseES V8R6 索引膨胀

索引膨胀

对于索引，随着业务不断的增删改，会造成膨胀，尤其Btree索引，也会涉及索引分裂、合并等，导致索引访问效率降低、维护成本增加。另外，索引页的复用与HEAP PAGE不一样，因为索引的内容是有序结构，只有符合顺序的ITEM才能插入对应的PAGE中，不像HEAP TUPLE，只要有空间就可以插入。index page无论在任何位置，都不能从磁盘删除，因此索引变大后，不能回收空间，除非vacuum full。所以索引膨胀后，通常需要重建索引来回收索引空间。

此外，对于B树索引，新构建的索引比多次更新的索引访问速度稍快，因为逻辑上相邻的页面通常在新构建索引中也是物理上相邻的。为了提高访问速度，定期重新B_Tree 索引可能是值得的。

对于重建索引，REINDEX在任何情况下都可以安全方便地使用。默认情况下，该命令需要ACCESS EXCLUSIVE锁。可以使用CONCURRENTLY选项创建索引，该选项只需要SHARE UPDATE EXCLUSIV锁，不阻塞读写。

索引膨胀的原因：

1.大量删除发生后，导致索引页面稀疏，降低了索引使用效率。

2.长时间运行的事务，阻止了vacuum对表的清理工作，因而导致页面稀疏状态一直保持。

3.索引字段乱序写入，导致索引频繁分裂，使得索引页并不是百分百填满，所以膨胀。

查询获取每个表的行数、索引以及有关这些索引的一些信息：

SELECT





    pg_class.relname,





    pg_size_pretty(pg_class.reltuples::bigint)            AS rows_in_bytes,





    pg_class.reltuples                                    AS num_rows,





    COUNT(*)                                              AS total_indexes,





    COUNT(*) FILTER ( WHERE indisunique)                  AS unique_indexes,





    COUNT(*) FILTER ( WHERE indnatts = 1 )                AS single_column_indexes,





    COUNT(*) FILTER ( WHERE indnatts IS DISTINCT FROM 1 ) AS multi_column_indexes





FROM





    pg_namespace





    LEFT JOIN pg_class ON pg_namespace.oid = pg_class.relnamespace





    LEFT JOIN pg_index ON pg_class.oid = pg_index.indrelid





WHERE





    pg_namespace.nspname = 'public' AND





    pg_class.relkind = 'r'





GROUP BY pg_class.relname, pg_class.reltuples





ORDER BY pg_class.reltuples DESC;

SELECT





    t.schemaname,





    t.tablename,





    c.reltuples::bigint                            AS num_rows,





    pg_size_pretty(pg_relation_size(c.oid))        AS table_size,





    psai.indexrelname                              AS index_name,





    pg_size_pretty(pg_relation_size(i.indexrelid)) AS index_size,





    CASE WHEN i.indisunique THEN 'Y' ELSE 'N' END  AS "unique",





    psai.idx_scan                                  AS number_of_scans,





    psai.idx_tup_read                              AS tuples_read,





    psai.idx_tup_fetch                             AS tuples_fetched





FROM





    pg_tables t





    LEFT JOIN pg_class c ON t.tablename = c.relname





    LEFT JOIN pg_index i ON c.oid = i.indrelid





    LEFT JOIN pg_stat_all_indexes psai ON i.indexrelid = psai.indexrelid





WHERE





    t.schemaname NOT IN ('pg_catalog', 'information_schema')





ORDER BY 1, 2;

查找重复索引，查找具有相同列集、相同操作类、表达式和谓词的多个索引，但需要人为判断需要删除的重复项：

SELECT pg_size_pretty(sum(pg_relation_size(idx))::bigint) as size,





       (array_agg(idx))[1] as idx1, (array_agg(idx))[2] as idx2,





       (array_agg(idx))[3] as idx3, (array_agg(idx))[4] as idx4





FROM (





    SELECT indexrelid::regclass as idx, (indrelid::text ||E'\n'|| indclass::text ||E'\n'|| indkey::text ||E'\n'||coalesce(indexprs::text,'')||E'\n' || coalesce(indpred::text,'')) as key





    FROM pg_index) sub





GROUP BY key HAVING count(*)>1





ORDER BY sum(pg_relation_size(idx)) DESC;

查找未使用的索引：

select 





    indexrelid::regclass as index, relid::regclass as table 





from 





    pg_stat_user_indexes 





    JOIN pg_index USING (indexrelid) 





where 





    idx_scan = 0 and indisunique is false;

有些场景，重建索引后，索引就变小了。通常这种情况是索引字段乱序写入，导致索引频繁分裂，使得索引页并不是百分百填满，密度低，索引页面浪费。

索引碎片模拟

乱序写入:





 





test=# create table t_split(id int);  





CREATE TABLE  





test=# create index idx_split on t_split (id);  





CREATE INDEX  





test=# insert into t_split select random()*1000000 from generate_series(1,1000000);  





INSERT 0 1000000  





test=# \di+ idx_split   





                          List of relations





 Schema |   Name    | Type  | Owner  |  Table  | Size  | Description





--------+-----------+-------+--------+---------+-------+-------------





 public | idx_split | index | system | t_split | 30 MB |





 





(1 row)





 





顺序写入:





 





test=# truncate t_split ;  





TRUNCATE TABLE  





test=# \di+ idx_split   





                            List of relations





 Schema |   Name    | Type  | Owner  |  Table  |    Size    | Description





--------+-----------+-------+--------+---------+------------+-------------





 public | idx_split | index | system | t_split | 8192 bytes |





 





(1 row)  





  





test=# insert into t_split select generate_series(1,1000000);  





INSERT 0 1000000  





test=# \di+ idx_split   





                          List of relations





 Schema |   Name    | Type  | Owner  |  Table  | Size  | Description





--------+-----------+-------+--------+---------+-------+-------------





 public | idx_split | index | system | t_split | 22 MB |





 





(1 row)





 





先写入数据，后建索引:





 





test=# drop index idx_split ;  





DROP INDEX  





test=# create index idx_split on t_split (id);  





CREATE INDEX  





test=# \di+ idx_split   





                           List of relations





 Schema |   Name    | Type  | Owner  |  Table  | Size  | Description





--------+-----------+-------+--------+---------+-------+-------------





 public | idx_split | index | system | t_split | 22 MB |





(1 row)





 





 





业务运行久了，不断的增删改，也会导致索引碎片:





 





test=# create table test(id int);





CREATE TABLE





test=# insert into test values(generate_series(1,1000000));





INSERT 0 1000000





test=# create index idx_fragmented on test(id);





CREATE INDEX





CREATE EXTENSION kbstattuple;





刚刚创建的索引没有碎片：





test=# \x





Expanded display is on.





test=# SELECT * FROM pgstatindex('idx_fragmented');





-[ RECORD 1 ]------+---------





version            | 4





tree_level         | 2





index_size         | 22609920





root_block_no      | 289





internal_pages     | 11





leaf_pages         | 2748





empty_pages        | 0





deleted_pages      | 0





avg_leaf_density   | 89.93





leaf_fragmentation | 0





 





 





leaf_fragmentation的碎片率是33.33%:





test=# insert into test values(generate_series(1,1000000));





INSERT 0 1000000





test=# SELECT * FROM pgstatindex('idx_fragmented');





-[ RECORD 1 ]------+---------





version            | 4





tree_level         | 2





index_size         | 67846144





root_block_no      | 289





internal_pages     | 39





leaf_pages         | 8242





empty_pages        | 0





deleted_pages      | 0





avg_leaf_density   | 60.06





leaf_fragmentation | 33.33





 





 





reindex之后，即可回收空间，减少碎片。





 





test=# reindex index idx_fragmented;





REINDEX





test=# \di+ idx_fragmented





List of relations





-[ RECORD 1 ]---------------





Schema      | public





Name        | idx_fragmented





Type        | index





Owner       | system





Table       | test





Size        | 43 MB





Description |





 





test=# SELECT * FROM pgstatindex('idx_fragmented');





-[ RECORD 1 ]------+---------





version            | 4





tree_level         | 2





index_size         | 45236224





root_block_no      | 208





internal_pages     | 26





leaf_pages         | 5495





empty_pages        | 0





deleted_pages      | 0





avg_leaf_density   | 90.01





leaf_fragmentation | 0

总结

通过以上方法监控索引膨胀，以及索引碎片情况，及时对索引reindex 进行碎片优化，建议不要在一个表上建太多索引，准确评估经常update的列和经常select的列，以便创建合适的索引。

文章知识点与官方知识档案匹配，可进一步学习相关知识

CS入门技能树Linux入门初识Linux32604 人正在系统学习中