postgresql双机热备、高可用方案（采用pacemaker+corosync实现）
http://blog.csdn.net/qguanri/article/details/51151974
需求描述
我们有两台centos7的数据库主机A、B。要对A、B实现双机热备，A作为数据库master主机，对外提供读写服务，B作为slave主机能实时同步A的数据。当A发生故障时，B从slave状态切换到master状态，以保证pg数据库能正常对外提供服务。
经过调研，最终我们给出的解决方案是采用pacemaker和corosync组件来实现postgresql的高可用。
所有部署脚本可以下github中找到：https://github.com/qinguanri/pg-ha
环境说明
- 操作系统centos7：
[root@10 ~]# cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
[root@10 ~]# uname -a
Linux 10.16.93.179 3.10.0-327.10.1.el7.x86_64 #1 SMP Tue Feb 16 17:03:50 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux- 1
- 2
- 3
- 4
postgresql版本：9.2.15
- 2台主机：
| 角色 | IP |
|---|---|
| node1 | 10.16.93.179 |
| node2 | 10.16.93.182 |
- 分配2个虚IP。我们的数据库对外提供服务时，暴露的是虚IP：
vip-master: 10.16.93.180
vip-slave: 10.16.93.181
配置Linux集群环境
注意：请严格按照下文所给的顺序进行配置
1.安装Pacemaker和Corosync包
需要在所有节点执行：
$ yum install -y pacemaker pcs psmisc policycoreutils-python postgresql-server- 1
2.禁用防火墙
在所有节点执行：
$ systemctl disable firewalld.service
$ systemctl stop firewalld.service- 1
- 2
3.启用pcs
在所有节点执行：
$ systemctl start pcsd.service
$ systemctl enable pcsd.service
ln -s '/usr/lib/systemd/system/pcsd.service' '/etc/systemd/system/multi-user.target.wants/pcsd.service'
$ echo hacluster | passwd hacluster --stdin
Changing password for user hacluster.
Changing password for user hacluster.
passwd: all authentication tokens updated successfully.- 1
- 2
- 3
- 4
- 5
- 6
- 7
4.集群认证
在任何一个节点上执行，这里选择node1：
$ pcs cluster auth -u hacluster -p hacluster 10.16.93.179 10.16.93.182
10.16.93.179: Authorized
10.16.93.182: Authorized- 1
- 2
- 3
5.同步配置
在node1上执行：
$ pcs cluster setup --last_man_standing=1 --name pgcluster 10.16.93.179 10.16.93.182
Shutting down pacemaker/corosync services...
Redirecting to /bin/systemctl stop pacemaker.service
Redirecting to /bin/systemctl stop corosync.service
Killing any remaining services...
Removing all cluster configuration files...
10.16.93.179: Succeeded
10.16.93.182: Succeeded- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
6.启动集群
在node1上执行。（有时需要稍等十几秒，才能看到命令的输出结果）：
$ pcs cluster start --all
172.17.5.90: Starting Cluster...
172.17.5.91: Starting Cluster...
172.17.5.92: Starting Cluster...- 1
- 2
- 3
- 4
7.检验
1）检验corosync
在node1上执行：
$ pcs status corosync
Membership information
----------------------
Nodeid Votes Name
1 1 10.16.93.179 (local)
2 1 10.16.93.182- 1
- 2
- 3
- 4
- 5
- 6
2）检验pacemaker
root@10 data]# pcs status
Cluster name: pgcluster
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Last updated: Mon Apr 4 23:45:48 2016 Last change: Fri Apr 1 09:17:03 2016 by root via crm_attribute on tsung_test
Stack: corosync
Current DC: tsung_test (version 1.1.13-10.el7_2.2-44eb2dd) - partition with quorum
2 nodes and 5 resources configured
Online: [ 10.16.93.179 tsung_test ]
Full list of resources:
Master/Slave Set: pgsql-cluster [pgsql]
Masters: [ tsung_test ]
Slaves: [ 10.16.93.179 ]
Resource Group: master-group
vip-master (ocf::heartbeat:IPaddr2): Started tsung_test
Resource Group: slave-group
vip-slave (ocf::heartbeat:IPaddr2): Started 10.16.93.179
PCSD Status:
10.16.93.179: Online
tsung_test (10.16.93.182): Online
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
注意：当vip-master和vip-slave都是Started状态时，集群节点配置正确。否则，请检查你的配置是否有误
安装和配置PostgreSQL
1.创建数据库目录
需要在所有节点上执行：
$ mkdir -p /data/postgresql/{data,xlog_archive}
$ chown -R postgres:postgres /data/postgresql/
$ chmod 0700 /data/postgresql/data- 1
- 2
- 3
2.初始化db
在node1上执行：
$ su - postgres
$ initdb -D /data/postgresql/data/
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
fixing permissions on existing directory /data/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 32MB
creating configuration files ... ok
creating template1 database in /data/postgresql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
postgres -D /data/postgresql/data
or
pg_ctl -D /data/postgresql/data -l logfile start- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
3.修改配置文件
在node1上执行：
$ vim /data/postgresql/data/postgresql.conf
listen_addresses = '*'
wal_level = hot_standby
synchronous_commit = on
archive_mode = on
archive_command = 'cp %p /data/postgresql/xlog_archive/%f'
max_wal_senders=5
wal_keep_segments = 32
hot_standby = on
restart_after_crash = off
replication_timeout = 5000
wal_receiver_status_interval = 2
max_standby_streaming_delay = -1
max_standby_archive_delay = -1
synchronous_commit = on
restart_after_crash = off
hot_standby_feedback = on
$ vim /data/postgresql/data/pg_hba.conf
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
host all all 10.16.0.0/16 md5
host all all 10.18.0.0/16 md5
host replication all 10.16.0.0/16 md5- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
注意：pg_hba.conf文件用于配置postgresql的访问控制，例如：host all all 10.16.0.0/16 md5,表示允许10.16.0.0/16网段的主机访问所有用户、所有数据库，密码使用md5密文的方式。你需要根据你的实际网段进行设置。当出现访问权限错误时，请检查你的pg_hba.conf文件
4.启动
启动数据库，创建replicator用户。将用户postgres的密码修改为postgres（pg初始安装时，postgres用户的登录密码默认为空）
在node1上执行：
$ pg_ctl -D /data/postgresql/data/ start
server starting
[ 2015-10-16 08:51:31.451 UTC 53158 5620ba93.cfa6 1 0]LOG: redirecting log output to logging collector process
[ 2015-10-16 08:51:31.451 UTC 53158 5620ba93.cfa6 2 0]HINT: Future log output will appear in directory "pg_log".
$ psql -U postgres
psql (9.2.13)
Type "help" for help.
postgres=# create role replicator with login replication password '8d5e9531-3817-460d-a851-659d2e51ca99';
CREATE ROLE
postgres=# alter user postgres with password 'postgres';
postgres=# \q- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
5.制作slave
在node2上执行：
$ su - postgres
$ pg_basebackup -h 10.16.93.179 -U postgres -D /data/postgresql/data/ -X stream -P
could not change directory to "/home/wenhang.pan"
20127/20127 kB (100%), 1/1 tablespace
$ vim /data/postgresql/data/recovery.conf
standby_mode = 'on'
primary_conninfo = 'host=10.16.93.179 port=5432 user=replicator application_name=zhaopin-5-92 password=8d5e9531-3817-460d-a851-659d2e51ca99 keepalives_idle=60 keepalives_in
terval=5 keepalives_count=5'
restore_command = 'cp /data/postgresql/xlog_archive/%f %p'
recovery_target_timeline = 'latest'- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
6.启动slave
在node2上执行：
$ pg_ctl -D /data/postgresql/data/ start
pg_ctl: another server might be running; trying to start server anyway
server starting
-bash-4.2$ LOG: database system was interrupted while in recovery at log time 2015-10-16 08:19:07 GMT
HINT: If this has occurred more than once some data might be corrupted and you might need to choose an earlier recovery target.
LOG: entering standby mode
LOG: redo starts at 0/3000020
LOG: consistent recovery state reached at 0/30000E0
LOG: database system is ready to accept read only connections
LOG: streaming replication successfully connected to primary- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
7.查看集群状态
在node1上执行一下命令。这个时候，可看到node2是一个replication。同步状态为sync
[root@10 ~]# psql -U postgres
psql (9.2.15)
Type "help" for help.
postgres=# select * from pg_stat_replication ;
pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_start
| state | sent_location | write_location | flush_location | replay_location | sync_priority | sync_state
-------+----------+------------+------------------+--------------+-----------------+-------------+-----------------------
--------+-----------+---------------+----------------+----------------+-----------------+---------------+------------
17349 | 16384 | replicator | 10.16.93.182 | 10.16.93.182 | | 22690 | 2016-04-14 14:07:25.18
1297+08 | streaming | 0/9014508 | 0/9014508 | 0/9014508 | 0/9014508 | 1 | sync
(1 row)
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
8.停止PostgreSQL服务
在node1、node2上执行：
$ pg_ctl -D /data/postgresql/data/ -mi stop
waiting for server to shut down.... done
server stopped- 1
- 2
- 3
注意：在配置自动切换前，一定要先把node1、node2上的PostgreSQL服务停掉
配置自动切换
上面的配置完成了数据库的初始化。下面进行自动切换相关的配置。当发生故障时，master/slave身份实现自动切换。
1.配置
在node1执行：
1）配置项较多，我们先将配置步骤写到脚本里
[root@10 ~]# vim cluster_setup.sh
# 将cib配置保存到文件
pcs cluster cib pgsql_cfg
# 在pacemaker级别忽略quorum
pcs -f pgsql_cfg property set no-quorum-policy="ignore"
# 禁用STONITH
pcs -f pgsql_cfg property set stonith-enabled="false"
# 设置资源粘性，防止节点在故障恢复后发生迁移
pcs -f pgsql_cfg resource defaults resource-stickiness="INFINITY"
# 设置多少次失败后迁移
pcs -f pgsql_cfg resource defaults migration-threshold="3"
# 设置master节点虚ip
pcs -f pgsql_cfg resource create vip-master IPaddr2 ip="10.16.93.180" cidr_netmask="24" op start timeout="60s" interval="0s" on-fail="restart" op monitor timeout="60s" interval="10s" on-fail="restart" op stop timeout="60s" interval="0s" on-fail="block"
# 设置slave节点虚ip
pcs -f pgsql_cfg resource create vip-slave IPaddr2 ip="10.16.93.181" cidr_netmask="24" op start timeout="60s" interval="0s" on-fail="restart" op monitor timeout="60s" interval="10s" on-fail="restart" op stop timeout="60s" interval="0s" on-fail="block"
# 设置pgsql集群资源
# pgctl、psql、pgdata和config等配置根据自己的环境修改,node list填写节点的hostname，master_ip填写虚master_ip
pcs -f pgsql_cfg resource create pgsql pgsql pgctl="/usr/bin/pg_ctl" psql="/usr/bin/psql" pgdata="/data/postgresql/data/" config="/data/postgresql/data/postgresql.conf" rep_mode="sync" node_list="tsung_test 10.16.93.179" master_ip="10.16.93.180" repuser="replicator" primary_conninfo_opt="password=8d5e9531-3817-460d-a851-659d2e51ca99 keepalives_idle=60 keepalives_interval=5 keepalives_count=5" restore_command="cp /data/postgresql/xlog_archive/%f %p" restart_on_promote='true' op start timeout="60s" interval="0s" on-fail="restart" op monitor timeout="60s" interval="4s" on-fail="restart" op monitor timeout="60s" interval="3s" on-fail="restart" role="Master" op promote timeout="60s" interval="0s" on-fail="restart" op demote timeout="60s" interval="0s" on-fail="stop" op stop timeout="60s" interval="0s" on-fail="block"
# 设置master/slave模式，clone-max=2，两个节点
pcs -f pgsql_cfg resource master pgsql-cluster pgsql master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true
# 配置master ip组
pcs -f pgsql_cfg resource group add master-group vip-master
# 配置slave ip组
pcs -f pgsql_cfg resource group add slave-group vip-slave
# 配置master ip组绑定master节点
pcs -f pgsql_cfg constraint colocation add master-group with master pgsql-cluster INFINITY
# 配置启动master节点
pcs -f pgsql_cfg constraint order promote pgsql-cluster then start master-group symmetrical=false score=INFINITY
# 配置停止master节点
pcs -f pgsql_cfg constraint order demote pgsql-cluster then stop master-group symmetrical=false score=0
# 配置slave ip组绑定slave节点
pcs -f pgsql_cfg constraint colocation add slave-group with slave pgsql-cluster INFINITY
# 配置启动slave节点
pcs -f pgsql_cfg constraint order promote pgsql-cluster then start slave-group symmetrical=false score=INFINITY
# 配置停止slave节点
pcs -f pgsql_cfg constraint order demote pgsql-cluster then stop slave-group symmetrical=false score=0
# 把配置文件push到cib
pcs cluster cib-push pgsql_cfg- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
2）执行操作文件
$ sh cluster_setup.sh- 1
2.查看状态
1）查看cluster状态
在node1上执行：
[root@10 ~]# pcs status
Cluster name: pgcluster
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Last updated: Mon Apr 4 23:15:56 2016 Last change: Fri Apr 1 09:17:03 2016 by root via crm_attribute on tsung_test
Stack: corosync
Current DC: tsung_test (version 1.1.13-10.el7_2.2-44eb2dd) - partition with quorum
2 nodes and 5 resources configured
Online: [ 10.16.93.179 tsung_test ]
Full list of resources:
Master/Slave Set: pgsql-cluster [pgsql]
Masters: [ tsung_test ]
Slaves: [ 10.16.93.179 ]
Resource Group: master-group
vip-master (ocf::heartbeat:IPaddr2): Started tsung_test
Resource Group: slave-group
vip-slave (ocf::heartbeat:IPaddr2): Started 10.16.93.179
PCSD Status:
10.16.93.179: Online
tsung_test (10.16.93.182): Online
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
2）查看PostgreSQL集群状态
在node1上执行：
[root@tsung_test data]# psql -U postgres
psql (9.2.15)
Type "help" for help.
postgres=# select * from pg_stat_replication ;
pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_start | state | sent_location | write_locatio
n | flush_location | replay_location | sync_priority | sync_state
-------+----------+------------+------------------+--------------+-----------------+-------------+-------------------------------+-----------+---------------+--------------
--+----------------+-----------------+---------------+------------
19415 | 16384 | replicator | 10.16.93.182 | 10.16.93.182 | | 18289 | 2016-04-01 09:17:29.037197-04 | streaming | 0/40F0ED8 | 0/40F0ED8
| 0/40F0ED8 | 0/40F0ED8 | 0 | async
(1 row)- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
验证自动切换
参考文献
- http://clusterlabs.org/wiki/PgSQL_Replicated_Cluster
- http://m.oschina.net/blog/518928
- http://m.oschina.net/blog/519458?mType=Group
postgresql双机热备、高可用方案（采用pacemaker+corosync实现）的更多相关文章
- Mysql+Keepalived双主热备高可用操作记录
我们通常说的双机热备是指两台机器都在运行,但并不是两台机器都同时在提供服务.当提供服务的一台出现故障的时候,另外一台会马上自动接管并且提供服务,而且切换的时间非常短.MySQL双主复制,即互为Mast ...
- 使用Keepalived实现Nginx的自动重启及双主热备高可用
1.概述 之前我们使用Keepalived实现了Nginx服务的双机主备高可用,但是有几个问题没有解决,今天一起探讨一下. 1)在双机主备机制中,Keepalived服务如果宕了,会自动启用备机进行服 ...
- 采用pacemaker+corosync实现postgresql双机热备、高可用方案
环境说明 参照上章已完成postgresql流复制配置,并关闭postgres服务. su - postgres pg_ctl -D /data/postgresql/data/ stop -m fa ...
- mysql+keepalived 双主热备高可用
理论介绍:我们通常说的双机热备是指两台机器都在运行,但并不是两台机器都同时在提供服务.当提供服务的一台出现故障的时候,另外一台会马上自动接管并且提供服务,而且切换的时间非常短.MySQL双主复制,即互 ...
- Redis双机热备方案--转
http://luyx30.blog.51cto.com/1029851/1350832 参考资料: http://patrick-tang.blogspot.com/2012/06/redis-ke ...
- Redis+Keepalived高可用方案详细分析
背景 目前,Redis集群的官方方案还处在开发测试中,未集成到稳定版中.且目前官方开发中的Redis Cluster提供的功能尚不完善(可参考官方网站或http://www.redisdoc.com/ ...
- MySQL高可用方案－PXC环境部署记录
之前梳理了Mysql+Keepalived双主热备高可用操作记录,对于mysql高可用方案,经常用到的的主要有下面三种: 一.基于主从复制的高可用方案:双节点主从 + keepalived 一般来说, ...
- 云数据库Redis版256M双机热备款
云数据库Redis版是兼容Redis协议标准的.提供持久化的缓存式数据库服务,基于高可靠双机热备架构:全新推出的256M小规格款,适用于高QPS.小数据量业务,并支持免费全量迁移,完美服务于个人开发者 ...
- 使用sqlserver搭建高可用双机热备的Quartz集群部署【附源码】
一般拿Timer和Quartz相比较的,简直就是对Quartz的侮辱,两者的功能根本就不在一个层级上,如本篇介绍的Quartz强大的序列化机制,可以序列到 sqlserver,mysql,当然还可以在 ...
随机推荐
- CodeForces 450B Jzzhu and Sequences（矩阵快速幂）题解
思路: 之前那篇完全没想清楚,给删了,下午一上班突然想明白了. 讲一下这道题的大概思路,应该就明白矩阵快速幂是怎么回事了. 我们首先可以推导出 学过矩阵的都应该看得懂,我们把它简写成T*A(n-1)= ...
- lxml.etree去除子节点
去除etree中的某个子节点有两种方法: 1.parentnode.remove(node) 2.etree.strip_elements(html, 'element_name', with_tag ...
- C#SendMessage用法
C#SendMessage用法 分类: C#操作内存相关 2011-11-26 23:52 1255人阅读 评论(0) 收藏 举报 函数功能:该函数将指定的消息发送到一个或多个窗口.此函数为指定的窗口 ...
- BZOJ 3064 CPU监控
题目链接:CPU监控 学习一番线段树的历史标记- 这道题就是区间加法,区间赋值,要询问区间最大值 和 区间历史最大值的最大值. 然后这种题就是在现有标记的基础上多弄一套标记,维护这个点出现过的最大的标 ...
- bzoj 2818 gcd 线性欧拉函数
2818: Gcd Time Limit: 10 Sec Memory Limit: 256 MB[Submit][Status][Discuss] Description 给定整数N,求1< ...
- codeforces 356 div2 C.Bear and Prime 100 数学
C. Bear and Prime 100 time limit per test 1 second memory limit per test 256 megabytes input standar ...
- 详解java中CAS机制所导致的问题以及解决——内存顺序冲突
[CAS机制] 指的是CompareAndSwap或CompareAndSet,是一个原子操作,实现此机制的原子类记录着当前值的在内存中存储的偏移地址,将内存中的真实值V与旧的预期值A做比较,如果不一 ...
- Java Spring-JdbcTemplate
2017-11-10 22:55:45 Spring 对持久层技术支持 : JDBC : org.springframework.jdbc.core.JdbcTemplate Hibernate3.0 ...
- 将keras模型在django中应用时出现的小问题——ValueError: Tensor Tensor("dense_2/Softmax:0", shape=(?, 8), dtype=float32) is not an element of this graph.
本文原出处(感谢作者提供):https://zhuanlan.zhihu.com/p/27101000 将keras模型在django中应用时出现的小问题 王岳王院长 10 个月前 keras 一个做 ...
- 识别TLS加密恶意流量
利用背景流量数据(contexual flow data)识别TLS加密恶意流量 识别出加密流量中潜藏的安全威胁具有很大挑战,现已存在一些检测方法利用数据流的元数据来进行检测,包括包长度和到达间隔时间 ...