Barman安装及备份PostgreSQL

barman特点

零数据丢失备份。保证用户在只有一台备份服务器的情况下达到零数据丢失。
与备份服务器合作。允许备份服务器在与主服务器的流式复制不可用时，从barman获取wal文件。
可靠的监控集成。用于监控和创建有关Barman配置状态的报告。
可并行的备份和恢复。以更快的速度执行拷贝，加快了整个备份和恢复过程。
备份目录。允许用户在同一界面中列出、保留、删除、存档和恢复多个完整备份。
支持多个服务器。允许您从同一位置管理多个服务器的备份和恢复。
存储备份数据。存储来自多个PostgreSQL实例的备份数据，支持不同的版本。
完全热备份。从其他服务器执行PostgreSQL服务器的完全热备份，专门为备份和灾难恢复而设计。
增量备份和恢复。针对新的和修改过的数据生成不同的备份，从而加快处理速度。
保留策略。配置自定义备份保留策略以满足组织的需要。
远程恢复。执行PostgreSQL服务器备份的远程恢复。
Wal文件压缩。自定义归档和压缩WAL文件和备份。
备份信息。生成所有备份的常规和磁盘使用信息。
地理冗余。给DBA和系统管理员为PostgreSQL数据库的灾难恢复解决方案增加另一层保障。

barman安装

192.168.2.2 pg （postgresql server服务器）

192.168.2.3 barman （barman备份服务器）

postgresql版本11.2

barman版本2.9.1

yum安装postgresql

yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/pgdg-centos11-11-2.noarch.rpm

yum install -y postgresql11-server-11.2 postgresql11-contrib-11.2

/usr/pgsql-11/bin/postgresql-11-setup initdb  #执行数据库初始化

修改postgres.conf配置

listen_addresses = '*'

port = 5432

wal_level = replica  #11版本后改为replica

archive_mode = on

archive_command = 'cp %p /var/lib/pgsql/11/data/pg_archive/%f'

max_wal_senders = 10  #建议大于等于3

wal_keep_segments = 64

synchronous_standby_names = '*'

hot_standby = on

log_filename = 'postgresql-%Y%m%d_%H%M.log'

修改pg_hba.conf配置

local   all             all                                       trust

host    all             all               0.0.0.0/0               md5

host    all             all               ::1/128                 trust

host    replication     barman            0.0.0.0/0               md5

host    replication     streaming_barman  0.0.0.0/0               md5

创建pg_archive，并设置用户组

mkdir /var/lib/pgsql/11/data/pg_archive

chown postgres:postgres /var/lib/pgsql/11/data/pg_archive -R

启动postgresql

systemctl enable postgresql-11

systemctl start postgresql-11

创建barman和barman_streaming用户

su postgres

createuser -s -P barman

#设置密码123456

createuser -P --replication streaming_barman

#设置密码123456

#创建.pgpass

touch ~/.pgpass

#添加

pg:5432:*:streaming_barman:123456

pg:5432:*:barman:123456

#更改权限

chmod 600 ~/.pgpass

#验证

psql -c 'SELECT version()' -U barman -h pg postgres

yum安装barman

由于barman是python写的，且有些功能需要使用postgresql的工具，安装时会有些依赖

yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/postgresql11-libs-11.2-1PGDG.rhel7.x86_64.rpm

yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/python2-psycopg2-2.8.2-1.rhel7.x86_64.rpm

yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/barman-2.9-1.rhel7.noarch.rpm

yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/barman-cli-2.9-1.rhel7.noarch.rpm

修改barman的公共配置

vi /etc/barman.conf

#修改以下配置

parallel_jobs = 2   #并发数

immediate_checkpoint = true   #在执行back时是否会自动执行checkpoint

network_compression = true    #网络传输压缩

修改barman对postgresql server的私有配置

cd /etc/barman.d/

cp streaming-server.conf-template pg.conf

#将pg.conf中的[streaming]修改为[pg]

将192.168.2.2 pg添加到/etc/hosts中

barman使用

1.创建复制槽

barman receive-wal --create-slot pg

2.barman check

[root@25ff7df1b128 pg]# barman check pg

Server pg:

	PostgreSQL: OK

	is_superuser: OK

	PostgreSQL streaming: OK

	wal_level: OK

	replication slot: OK

	directories: OK

	retention policy settings: OK

	backup maximum age: OK (no last_backup_maximum_age provided)

	compression settings: OK

	failed backups: OK (there are 0 failed backups)

	minimum redundancy requirements: FAILED (have 0 backups, expected at least 1)

	pg_basebackup: OK

	pg_basebackup compatible: OK

	pg_basebackup supports tablespaces mapping: OK

	pg_receivexlog: OK

	pg_receivexlog compatible: OK

	receive-wal running: OK

	archiver errors: OK

其中minimum redundancy requirements: FAILED可忽略，首次配置后还没有全备，执行全备会变为OK

[root@25ff7df1b128 pg]# barman backup pg

Starting backup using postgres method for server pg in /var/lib/barman/pg/base/20190821T171724

Backup start at LSN: 0/90000D0 (000000010000000000000009, 000000D0)

Starting backup copy via pg_basebackup for 20190821T171724

Copy done (time: 2 seconds)

Finalising the backup.

This is the first backup for server pg

WAL segments preceding the current backup have been found:

	000000010000000000000008 from server pg has been removed

Backup size: 166.6 MiB

Backup end at LSN: 0/B000060 (00000001000000000000000B, 00000060)

Backup completed (start time: 2019-08-21 17:17:24.956763, elapsed time: 2 seconds)

Processing xlog segments from streaming for pg

	000000010000000000000009

	00000001000000000000000A

首次配置完会产生minimum redundancy requirements: FAILED、receive-wal running:FAILED，执行barman cron即可变为OK，barman会执行如下过程

[root@25ff7df1b128 pg]# barman cron

Starting WAL archiving for server pg

Starting streaming archiver for server pg

Starting check-backup for backup 20190821T172217 of server pg

#正常流复制时的进程

[root@25ff7df1b128 pg]# ps -ef|grep barman

root      1278     1  0 17:30 ?        00:00:00 /usr/bin/python2 /usr/bin/barman -c /etc/barman.conf -q receive-wal pg

root      1281  1278  0 17:30 ?        00:00:00 /usr/pgsql-11/bin/pg_receivewal --dbname=dbname=replication host=pg options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_receive_wal --verbose --no-loop --no-password --directory=/var/lib/barman/pg/streaming --slot=barman --synchronous

#执行全备时的进程

[root@25ff7df1b128 ~]# ps -ef|grep barman

root      1325     1  1 17:33 ?        00:00:00 /usr/bin/python2 /usr/bin/barman -c /etc/barman.conf -q receive-wal pg

root      1327  1325  0 17:33 ?        00:00:00 /usr/pgsql-11/bin/pg_receivewal --dbname=dbname=replication host=pg options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_receive_wal --verbose --no-loop --no-password --directory=/var/lib/barman/pg/streaming --slot=barman --synchronous

root      1331  1035 12 17:33 pts/0    00:00:00 /usr/bin/python2 /usr/bin/barman backup pg

root      1334  1331  0 17:33 pts/0    00:00:00 /usr/pgsql-11/bin/pg_basebackup --dbname=dbname=replication host=pg options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_streaming_backup -v --no-password --pgdata=/var/lib/barman/pg/base/20190821T173331/data --no-slot --wal-method=none --checkpoint=fast

所以barman的全备是利用pg_basebackup实现，增量是利用pg_receivewal实现。

barman优点

1.可以与postgresql的master和slaver分离部署，但是barman服务器需要安装postgresql的一些工具，如pg_basebackup、pg_receivewal

2.通过流复制可以同步拉取wal，如果postgresql当前的wal还没close，barman会产生一个.partial文件实时同步，若在恢复时使用该文件需要将扩展后缀去掉

3.barman可以一台服务器管理多个postgresql server，提供本地及远程恢复（远程恢复需要配置ssh免密）

4.提供备份相关文件的检测及详细查询

barman部分功能

1.查看当前复制状态

[root@25ff7df1b128 ~]# barman replication-status pg

Status of streaming clients for server 'pg':

  Current LSN on master: 0/130001A8

  Number of streaming clients: 1

  1. #1 Sync WAL streamer

     Application name: barman_receive_wal

     Sync stage      : 3/3 Remote write

     Communication   : TCP/IP

     IP Address      : 192.168.2.3 / Port: 33256 / Host: -

     User name       : streaming_barman

     Current state   : streaming (sync)

     Replication slot: barman

     WAL sender PID  : 1731

     Started at      : 2019-08-21 17:33:16.495522+08:00

     Sent LSN   : 0/130001A8 (diff: 0 B)

     Write LSN  : 0/130001A8 (diff: 0 B)

     Flush LSN  : 0/130001A8 (diff: 0 B)

2.查看备份列表

[root@25ff7df1b128 ~]# barman list-backup pg

pg 20190821T173331 - Wed Aug 21 17:33:35 2019 - Size: 294.6 MiB - WAL Size: 0 B - WAITING_FOR_WALS

pg 20190821T172217 - Wed Aug 21 17:22:20 2019 - Size: 230.6 MiB - WAL Size: 64.0 MiB

pg 20190821T171724 - Wed Aug 21 17:17:27 2019 - Size: 198.6 MiB - WAL Size: 32.0 MiB

3.查看postgresql server

[root@25ff7df1b128 ~]# barman show-server pg

Server pg:

	active: True

	archive_timeout: 3600

	archiver: False

	archiver_batch_size: 0

	backup_directory: /var/lib/barman/pg

	backup_method: postgres

	backup_options: BackupOptions(['concurrent_backup'])

	bandwidth_limit: None

	barman_home: /var/lib/barman

	barman_lock_directory: /var/lib/barman

	basebackup_retry_sleep: 30

	basebackup_retry_times: 3

	basebackups_directory: /var/lib/barman/pg/base

	check_timeout: 30

	checkpoint_timeout: 300

	compression: None

	config_file: /var/lib/pgsql/11/data/postgresql.conf

	connection_error: None

	conninfo: host=pg user=barman dbname=postgres

	current_size: 23808077

	current_xlog: 000000010000000000000015

	custom_compression_filter: None

	custom_decompression_filter: None

	data_checksums: off

	data_directory: /var/lib/pgsql/11/data

	description: Example of PostgreSQL Database (Streaming-Only)

	disabled: False

	errors_directory: /var/lib/barman/pg/errors

	hba_file: /var/lib/pgsql/11/data/pg_hba.conf

	hot_standby: on

	ident_file: /var/lib/pgsql/11/data/pg_ident.conf

	immediate_checkpoint: True

	incoming_wals_directory: /var/lib/barman/pg/incoming

	is_in_recovery: False

	is_superuser: True

	last_backup_maximum_age: None

	max_incoming_wals_queue: None

	max_replication_slots: 10

	max_wal_senders: 10

	minimum_redundancy: 1

	msg_list: []

	name: pg

	network_compression: False

	parallel_jobs: 1

	passive_node: False

	path_prefix: /usr/pgsql-11/bin

	pg_basebackup_bwlimit: True

	pg_basebackup_compatible: True

	pg_basebackup_installed: True

	pg_basebackup_path: /usr/pgsql-11/bin/pg_basebackup

	pg_basebackup_tbls_mapping: True

	pg_basebackup_version: 11.2

	pg_receivexlog_compatible: True

	pg_receivexlog_installed: True

	pg_receivexlog_path: /usr/pgsql-11/bin/pg_receivewal

	pg_receivexlog_supports_slots: True

	pg_receivexlog_synchronous: True

	pg_receivexlog_version: 11.2

	pgespresso_installed: False

	post_archive_retry_script: None

	post_archive_script: None

	post_backup_retry_script: None

	post_backup_script: None

	post_delete_retry_script: None

	post_delete_script: None

	post_recovery_retry_script: None

	post_recovery_script: None

	post_wal_delete_retry_script: None

	post_wal_delete_script: None

	pre_archive_retry_script: None

	pre_archive_script: None

	pre_backup_retry_script: None

	pre_backup_script: None

	pre_delete_retry_script: None

	pre_delete_script: None

	pre_recovery_retry_script: None

	pre_recovery_script: None

	pre_wal_delete_retry_script: None

	pre_wal_delete_script: None

	primary_ssh_command: None

	recovery_options: RecoveryOptions([])

	replication_slot: Record(slot_name='barman', active=True, restart_lsn='0/150000C8')

	replication_slot_support: True

	retention_policy: None

	retention_policy_mode: auto

	reuse_backup: None

	server_txt_version: 11.2

	slot_name: barman

	ssh_command: None

	streaming: True

	streaming_archiver: True

	streaming_archiver_batch_size: 0

	streaming_archiver_name: barman_receive_wal

	streaming_backup_name: barman_streaming_backup

	streaming_conninfo: host=pg user=streaming_barman

	streaming_supported: True

	streaming_wals_directory: /var/lib/barman/pg/streaming

	synchronous_standby_names: ['*']

	systemid: 6727448561719488729

	tablespace_bandwidth_limit: None

	timeline: 1

	wal_compression: off

	wal_level: replica

	wal_retention_policy: main

	wals_directory: /var/lib/barman/pg/wals

	xlog_segment_size: 16777216

	xlogpos: 0/150000C8

4.远程执行switch-wal

[root@25ff7df1b128 ~]# barman switch-wal pg

The WAL file 000000010000000000000015 has been closed on server 'pg'

5.检查backupid

[root@25ff7df1b128 ~]# barman list-backup pg

pg 20190821T174304 - Wed Aug 21 17:43:08 2019 - Size: 326.6 MiB - WAL Size: 0 B - WAITING_FOR_WALS

pg 20190821T173331 - Wed Aug 21 17:33:35 2019 - Size: 310.6 MiB - WAL Size: 16.0 MiB - WAITING_FOR_WALS

pg 20190821T172217 - Wed Aug 21 17:22:20 2019 - Size: 230.6 MiB - WAL Size: 80.0 MiB

pg 20190821T171724 - Wed Aug 21 17:17:27 2019 - Size: 198.6 MiB - WAL Size: 32.0 MiB

[root@25ff7df1b128 ~]# barman check-backup pg 20190821T171724

[root@25ff7df1b128 ~]# barman check-backup pg 20190821T171724s

ERROR: Unknown backup '20190821T171724s' for server 'pg'

6.查看文件

–target参数包括：

data：列出data目录的所有文件

standalone：列出全备和需要的wal文件

wal：列出全备开始到当前时间的wal文件

full：等于data + wal

[root@25ff7df1b128 ~]# barman list-files --target wal pg 20190821T173331

/var/lib/barman/pg/wals/0000000100000000/000000010000000000000012

/var/lib/barman/pg/wals/0000000100000000/000000010000000000000013

7.备份恢复

#备份恢复

barman recover <server_name> <backup_id> <postgres server data path>

#时间点恢复

barman recover <server_name> <backup_id> <postgres server data path> --target-time TARGET_TIME

还提供：--target-xid TARGET_XID、--target-lsn TARGET_LSN、--target-name TARGET_NAME、--target-immediate

8.查看备份详情

[root@25ff7df1b128 ~]# barman show-backup pg 20190821T173331

Backup 20190821T173331:

  Server Name            : pg

  Status                 : WAITING_FOR_WALS

  PostgreSQL Version     : 110002

  PGDATA directory       : /var/lib/pgsql/11/data

  Base backup information:

    Disk usage           : 294.6 MiB (310.6 MiB with WALs)

    Incremental size     : 294.6 MiB (-0.00%)

    Timeline             : 1

    Begin WAL            : 000000010000000000000012

    End WAL              : 000000010000000000000012

    WAL number           : 1

    Begin time           : 2019-08-21 17:33:31+08:00

    End time             : 2019-08-21 17:33:35.321271+08:00

    Copy time            : 3 seconds

    Estimated throughput : 75.6 MiB/s

    Begin Offset         : 96

    End Offset           : 0

    Begin LSN           : 0/12000060

    End LSN             : 0/13000000

  WAL information:

    No of files          : 1

    Disk usage           : 16.0 MiB

    WAL rate             : 12.58/hour

    Last available       : 000000010000000000000013

  Catalog information:

    Retention Policy     : not enforced

    Previous Backup      : 20190821T172217

    Next Backup          : 20190821T174304

参考：