Postgres 9.4 feature highlight: REPLICA IDENTITY and logical replication
Among the many things to say about logical replication features added in PostgreSQL 9.4, REPLICA IDENTITY is a new table-level parameter that can be used to control the information written to WAL to identify tuple data that is being deleted or updated (an update being a succession of an insert and a delete in MVCC).
This parameter has 4 modes:
- DEFAULT
- USING INDEX index
- FULL
- NOTHING
First let's set up an environment using some of the instructions in a previous post dealing with some basics of logical decoding to set up a server using test_decoding in a replication slot.
=# SELECT * FROM pg_create_logical_replication_slot('my_slot', 'test_decoding');
slot_name | xlog_position
-----------+---------------
my_slot | 0/16CB0F8
(1 row)
The replication slot used here will be used in combination with pg_logical_slot_get_changes to consume each change of the slot (to compare with pg_logical_slot_peek_changes that can be used to view the changes but not consume them).
In the case of DEFAULT, old tuple data is only identified with the primary key of the table. This data is written into WAL only when at least one column of the primary key is updated. Columns that are not part of the primary key do not have their old value written.
=# CREATE TABLE aa (a int, b int, c int, PRIMARY KEY (a, b));
CREATE TABLE
=# INSERT INTO aa VALUES (1,1,1);
INSERT 0 1
=# [ ... Clean up of slot information up to now ... ]
=# UPDATE aa SET c = 3 WHERE (a, b) = (1, 1);
UPDATE 1
=# SELECT * FROM pg_logical_slot_get_changes('my_slot', NULL, NULL);
location | xid | data
-----------+------+-----------------------------------------------------------------
0/1728D50 | 1013 | BEGIN 1013
0/1728D50 | 1013 | table public.aa: UPDATE: a[integer]:1 b[integer]:1 c[integer]:3
0/1728E70 | 1013 | COMMIT 1013
(3 rows)
=# UPDATE aa SET a = 2 WHERE (a, b) = (1, 1);
UPDATE 1
=# SELECT * FROM pg_logical_slot_get_changes('my_slot', NULL, NULL);
location | xid | data
-----------+------+---------------------------------------------------------------------------------------------------------------
0/1728EA8 | 1014 | BEGIN 1014
0/1728EA8 | 1014 | table public.aa: UPDATE: old-key: a[integer]:1 b[integer]:1 new-tuple: a[integer]:2 b[integer]:1 c[integer]:3
0/1728FF0 | 1014 | COMMIT 1014
(3 rows)
Ît is important to know that REPLICA IDENTITY can only be changed using ALTER TABLE, and that the parameter value is only viewable with '\d+' only if default behavior is not used. Also, after creating a table, REPLICA IDENTITY is set to DEFAULT (Surprise!).
=# \d+ aa
Table "public.aa"
Column | Type | Modifiers | Storage | Stats target | Description
--------+---------+-----------+---------+--------------+-------------
a | integer | not null | plain | |
b | integer | not null | plain | |
c | integer | | plain | |
Indexes:
"aa_pkey" PRIMARY KEY, btree (a, b)
=# ALTER TABLE aa REPLICA IDENTITY FULL;
ALTER TABLE
=# \d+ aa
Table "public.aa"
Column | Type | Modifiers | Storage | Stats target | Description
--------+---------+-----------+---------+--------------+-------------
a | integer | not null | plain | |
b | integer | not null | plain | |
c | integer | | plain | |
Indexes:
"aa_pkey" PRIMARY KEY, btree (a, b)
Replica Identity: FULL
=# [ ... Replication slot changes are consumed here ... ]
In the case of FULL, all the column values are written to WAL all the time. This is the most verbose, and as well the most resource-consuming mode. Be careful here particularly for heavily-updated tables.
=# UPDATE aa SET c = 4 WHERE (a, b) = (2, 1);
UPDATE 1
=# SELECT * FROM pg_logical_slot_get_changes('my_slot', NULL, NULL);
location | xid | data
-----------+------+----------------------------------------------------------------------------------------------------------------------------
0/172EC70 | 1016 | BEGIN 1016
0/172EC70 | 1016 | table public.aa: UPDATE: old-key: a[integer]:2 b[integer]:1 c[integer]:3 new-tuple: a[integer]:2 b[integer]:1 c[integer]:4
0/172EE00 | 1016 | COMMIT 1016
On the contrary, NOTHING prints... Nothing. (Note: operation done after an appropriate ALTER TABLE and after consuming replication slot information).
=# UPDATE aa SET c = 4 WHERE (a, b) = (2, 1);
UPDATE 1
=# SELECT * FROM pg_logical_slot_get_changes('my_slot', NULL, NULL);
location | xid | data
-----------+------+-----------------------------------------------------------------
0/1730F58 | 1018 | BEGIN 1018
0/1730F58 | 1018 | table public.aa: UPDATE: a[integer]:2 b[integer]:1 c[integer]:4
0/1731100 | 1018 | COMMIT 1018
Finally, there is USING INDEX, which writes to WAL the values of the index defined with this option. The index needs to be unique, cannot contain expressions and must contain NOT NULL columns.
=# ALTER TABLE aa ALTER COLUMN c SET NOT NULL;
ALTER TABLE
=# CREATE unique INDEX aai on aa(c);
CREATE INDEX
=# ALTER TABLE aa REPLICA IDENTITY USING INDEX aai;
ALTER TABLE
=# [ ... Consuming all information from slot ... ]
=# UPDATE aa SET c = 5 WHERE (a, b) = (2, 1);
UPDATE 1
=# SELECT * FROM pg_logical_slot_get_changes('my_slot', NULL, NULL);
location | xid | data
-----------+------+--------------------------------------------------------------------------------------------------
0/1749A68 | 1029 | BEGIN 1029
0/1749A68 | 1029 | table public.aa: UPDATE: old-key: c[integer]:4 new-tuple: a[integer]:2 b[integer]:1 c[integer]:5
0/1749D40 | 1029 | COMMIT 1029
(3 rows)
Note that in this case the primary key information is not decoded, only the NOT NULL column c that the index covers.
REPLICA IDENTITY should be chosen carefully for each table of a given application, knowing that for example FULL generates an extra amount of WAL that may not be necessary, NOTHING may forget about essential information. In most of the cases, DEFAULT provides a good cover though.
REPLICA IDENTITY
This form changes the information which is written to the write-ahead log to identify rows which are updated or deleted. This option has no effect except when logical replication is in use. DEFAULT (the default for non-system tables) records the old values of the columns of the primary key, if any. USING INDEX records the old values of the columns covered by the named index, which must be unique, not partial, not deferrable, and include only columns marked NOT NULL. FULL records the old values of all columns in the row. NOTHING records no information about the old row. (This is the default for system tables.) In all cases, no old values are logged unless at least one of the columns that would be logged differs between the old and new versions of the row.
参考:
http://michael.otacoo.com/postgresql-2/postgres-9-4-feature-highlight-replica-identity-logical-replication/
http://www.postgresql.org/docs/devel/static/sql-altertable.html
Postgres 9.4 feature highlight: REPLICA IDENTITY and logical replication的更多相关文章
- 【ASP.NET Identity系列教程(一)】ASP.NET Identity入门
注:本文是[ASP.NET Identity系列教程]的第一篇.本系列教程详细.完整.深入地介绍了微软的ASP.NET Identity技术,描述了如何运用ASP.NET Identity实现应用程序 ...
- ASP.NET Identity 一 (转载)
来源:http://www.cnblogs.com/r01cn/p/5194257.html 注:本文是[ASP.NET Identity系列教程]的第一篇.本系列教程详细.完整.深入地介绍了微软的A ...
- Postgres 主从复制搭建步骤
系统版本: CentOS Linux release 7.5.1804 (Core) 数据库 psql (PostgreSQL) 10.5 2台机器ip : 172.17.0.3 /172.17.0. ...
- ASP.NET Identity系列教程-2【Identity入门】
https://www.cnblogs.com/r01cn/p/5177708.html13 Identity入门 Identity is a new API from Microsoft to ma ...
- flink-cdc读取postgres报异常,没有发布表
异常信息 must be superuser to create FOR ALL TABLES publication 必须是超级用户才能为所有发布表创建 网上搜索了一天,都毫无头绪,后面搜索到了一个 ...
- TSQL Identity 用法全解
Identity是标识值,在SQL Server中,有ID列,ID属性,ID值,ID列的值等术语. Identity属性是指在创建Table时,为列指定的Identity属性,其语法是:column_ ...
- mongodb复制集Replica Set使用简介
MongoDB高可用 对于MongoDB,可以支持使用单机模式提供服务,但是在实际的生产环境中,单机模式将面临很大的风险,一旦这个数据库服务出现问题,就会导致线上的服务出现错误甚至崩溃.因此,在实际生 ...
- Replication Controller 和 Replica Set
使用Replication Controller . Replica Set管理Pod Replication Controller (RC) 简写为RC,可以使用rc作为kubectl工具的快速管理 ...
- postgres使用pg_ctl 命令
想要用pg_ctl等一系列的命令,需要配置环境变量: PATH=$PATH:$HOME/.local/bin:$HOME/bin:/usr/local/pgsql/binexport PGDATA=/ ...
随机推荐
- java高薪之路__005_IO流
参考地址: 1. http://blog.csdn.net/yczz/article/details/38761237 File类 ObjectInputStream && Objec ...
- 程序设计入门——C语言 第3周编程练习 2 数字特征值(5分)
2 数字特征值(5分) 题目内容: 对数字求特征值是常用的编码算法,奇偶特征是一种简单的特征值.对于一个整数,从个位开始对每一位数字编号,个位是1号,十位是2号,以此类推.这个整数在第n位上的数字记作 ...
- (八)open函数的flag详解
3.1.4.open函数的flag详解13.1.4.1.读写权限:O_RDONLY O_WRONLY O_RDWR(1)linux中文件有读写权限,我们在open打开文件时也可以附带一定的权限说明(譬 ...
- ThreadLocal的分享
最开始的时候打算自己写点什么,但是看了这些博客以后感觉真的不知道应该写点什么了,全部都是好文章,只做分享了,链接如下: 1.http://www.cnblogs.com/dolphin0520/p/3 ...
- [转载]【基础篇】不为人知的Maya移动坐标轴
maya 设置轴向1.将整体模型方向调整到与世界坐标系一致.设置具体模型的轴朝向. 操作:选中模型,按住W键,鼠标左键,在弹出的菜单中选择Axis,比较常用的有Set To Ponit.Set To ...
- 防止SVN冲突,Elipse资源同步介绍
灰色向右箭头: 本地修改了 灰色向右箭头且中间有白色减号: 本地删除了,服务器未删除 灰色向右且中间有个加号的箭头:本地比SVN上多出的文件 蓝色向左箭头:svn上修改过 蓝色向左且中间有个加号的箭头 ...
- TOSHIBA TEC EXT Printer Z-Mode
Z-Mode functionality automatically converts the Zebra data stream into a TOSHIBA data stream (TPCL). ...
- jquery插件开发基础入门
jquery插件开发基础入门 入门 编写一个jquery插件开始于给jquery.fn加入新的功能属性,此处添加的对象属性的名称就是你的插件名称 jQuery.fn,myPlugin = functi ...
- 关于OPencv版本不符合,相关库变化问题
由于OPencv发展迅速,已经省略了很多原来的库文件,奈何自己才疏学浅,所以只能把OPencv 1.0中的相关版本中的库文件一直过去. 链接: http://pan.baidu.com/s/1qY1Z ...
- PDF 补丁丁 0.4.2.950 测试版发布:按文件夹合并生成单独的PDF文件
新的测试版实现了将文件夹的内容合并为单独的PDF文件的功能.以下图为例讲解操作步骤. 点击工具栏的“合并文件”按钮,打开合并功能. 设M盘下有“test”和“test2”文件夹,里面包含了需要合并的文 ...