Openstack的HA解决方案【mysql集群配置】

使用mysql的galera做多主集群配置，galera的集群优势网络上面有对比，这里不在叙述。

1. 新建3台虚拟机（centos6.5)

  node1:172.17.44.163
  node2:172.17.44.164
  node3:172.17.44.165
  他们没有主从关系，所有配置都应该一样。

2. 建立软件源

  我们有一个内部的软件yum 源,至于怎么建立源服务器，可参考网络上的介绍，我们的源文件的nailgun.repo

[nailgun]
name=Nailgun
baseurl=http://172.17.44.2:8080/centos/fuelweb/x86_64/
gpgcheck=0

 将这个文件放在/etc/yum.repo.d/目录下

3. 搜索和mysql galera有关的软件安装包 执行结果：

[root@ha1 yum.repos.d]# yum search wsrep
Loaded plugins: fastestmirror
Repository base is listed more than once in the configuration
Repository updates is listed more than once in the configuration
Repository extras is listed more than once in the configuration
Repository centosplus is listed more than once in the configuration
Repository contrib is listed more than once in the configuration
Loading mirror speeds from cached hostfile
 * base: centos.ustc.edu.cn
 * epel: mirrors.ustc.edu.cn
 * extras: centos.ustc.edu.cn
 * updates: centos.ustc.edu.cn
===================================================================================== N/S Matched: wsrep ======================================================================================
MySQL-client-wsrep.x86_64 : MySQL - Client
MySQL-server-wsrep.x86_64 : MySQL: a very fast and reliable SQL database server
MySQL-shared-wsrep.x86_64 : MySQL - Shared libraries
galera.x86_64 : Galera: a synchronous multi-master wsrep provider (replication engine)

  Name and summary matches only, use "search all" for everything.
[root@ha1 yum.repos.d]#

主要就是安装这4个包。

4.安装后基本配置： 安装后正常应该会有/etc/my.cnf文件。但是不够，因为my.cnf文件是普通版本的mysql配置，而galera的mysql需要更多配置。

a.增加目录
  <pre>
    mkdir -p /etc/mysql/conf.d
  </pre>

b.在/etc/mysql/conf.d/添加文件wsrep.cnf
  内容是:

[mysqld]
binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
innodb_locks_unsafe_for_binlog=1
query_cache_size=0
query_cache_type=0
bind-address=172.17.44.163
port=3307
max_connections=2048
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_provider_options="pc.ignore_sb = no;ist.recv_addr=172.17.44.163;gmcast.listen_addr=tcp://172.17.44.163:4567;gcache.size=1G"
wsrep_cluster_name="openstack"
wsrep_cluster_address="gcomm://172.17.44.164:4567,172.17.44.164:4567,172.17.44.165:4567?pc.wait_prim=no"
wsrep_node_address=172.17.44.163
wsrep_slave_threads=4
wsrep_certify_nonPK=1
wsrep_max_ws_rows=131072
wsrep_max_ws_rows=131072
wsrep_max_ws_size=1073741824
wsrep_debug=0
wsrep_convert_LOCK_to_trx=0
wsrep_retry_autocommit=1
wsrep_auto_increment_control=1
wsrep_drupal_282555_workaround=0
wsrep_causal_reads=0
wsrep_notify_cmd=
wsrep_sst_method=mysqldump
wsrep_sst_receive_address=172.17.44.163:3307
wsrep_sst_auth=wsrep_sst:password
skip-name-resolve
innodb_buffer_pool_size=563M
innodb_thread_concurrency=0
innodb_write_io_threads=8
innodb_read_io_threads=8
innodb_io_capacity=500
table_cache=10000
innodb_file_per_table=1
innodb_file_format=Barracuda

3台机器都应该有上面的配置。需要对IP进行部分修改，上面配置的IP是44.163机器的，如果是164，需要将bind-address，wsrep_sst_receive_address做出修改。

c.修改my.cnf文件

[mysqld_safe]
syslog
!includedir /etc/mysql/conf.d/

其他都注释.

5.启动集群。

第一次启动是有问题的。因为我们还没有设置我们的集群底层用户,sst_auth,主要在配置文件wsrep_sst_auth选项。我们需要给每一台主机都设置上这么一个用户才可以。
我们需要先将wsrep_cluster_address="gcomm://172.17.44.164:4567,172.17.44.164:4567,172.17.44.165:4567?pc.wait_prim=no"注释（所有机器），改为

wsrep_cluster_address="gcomm://",即当前机器是一个单一的mysql galera

然后创建用户：

grant all on *.* to wsrep_sst@"%" identified by "password";
grant all on *.* to wsrep_sst@"localhost" identified by "password";

创建后测试该用户是否可用。注意我们的端口使用的是3307.

然后将配置wsrep_cluster_address改到所有用户，重新启动mysql galera.然后进入mysql，查看其状态：

mysql> show status like "wsrep_%";
+----------------------------+----------------------------------------------------------+
| Variable_name              | Value                                                    |
+----------------------------+----------------------------------------------------------+
| wsrep_local_state_uuid     | f0e10cb3-249a-11e4-0800-2666f281655b                     |
| wsrep_protocol_version     | 4                                                        |
| wsrep_last_committed       | 4                                                        |
| wsrep_replicated           | 0                                                        |
| wsrep_replicated_bytes     | 0                                                        |
| wsrep_received             | 23                                                       |
| wsrep_received_bytes       | 2118                                                     |
| wsrep_local_commits        | 0                                                        |
| wsrep_local_cert_failures  | 0                                                        |
| wsrep_local_bf_aborts      | 0                                                        |
| wsrep_local_replays        | 0                                                        |
| wsrep_local_send_queue     | 0                                                        |
| wsrep_local_send_queue_avg | 0.000000                                                 |
| wsrep_local_recv_queue     | 0                                                        |
| wsrep_local_recv_queue_avg | 0.000000                                                 |
| wsrep_flow_control_paused  | 0.000000                                                 |
| wsrep_flow_control_sent    | 0                                                        |
| wsrep_flow_control_recv    | 0                                                        |
| wsrep_cert_deps_distance   | 1.000000                                                 |
| wsrep_apply_oooe           | 0.000000                                                 |
| wsrep_apply_oool           | 0.000000                                                 |
| wsrep_apply_window         | 0.000000                                                 |
| wsrep_commit_oooe          | 0.000000                                                 |
| wsrep_commit_oool          | 0.000000                                                 |
| wsrep_commit_window        | 0.000000                                                 |
| wsrep_local_state          | 4                                                        |
| wsrep_local_state_comment  | Synced                                                   |
| wsrep_cert_index_size      | 2                                                        |
| wsrep_causal_reads         | 0                                                        |
| wsrep_incoming_addresses   | 172.17.44.163:3307,172.17.44.164:3307,172.17.44.165:3307 |
| wsrep_cluster_conf_id      | 13                                                       |
| wsrep_cluster_size         | 3                                                        |
| wsrep_cluster_state_uuid   | f0e10cb3-249a-11e4-0800-2666f281655b                     |
| wsrep_cluster_status       | Primary                                                  |
| wsrep_connected            | ON                                                       |
| wsrep_local_index          | 0                                                        |
| wsrep_provider_name        | Galera                                                   |
| wsrep_provider_vendor      | Codership Oy <info@codership.com>                        |
| wsrep_provider_version     | 23.2.2(r137)                                             |
| wsrep_ready                | ON                                                       |
+----------------------------+----------------------------------------------------------+
40 rows in set (0.00 sec)

这里的incoming_addresses地址是3个，wsrep_ready 为ON就表示正常了。

不正常原因。 1. 网络防火墙。 2. 用户权限。 3. 其他。

我们来看一下我们的成果。 基本测试就是一台机器上创建一个数据库，要再其他机器上也能看到。 163的：

| wsrep_provider_version     | 23.2.2(r137)                                             |
| wsrep_ready                | ON                                                       |
+----------------------------+----------------------------------------------------------+
40 rows in set (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| aa                 |
| mysql              |
| performance_schema |
| tangcl             |
| test               |
+--------------------+
6 rows in set (0.00 sec)

mysql>

164的：

mysql> show databses;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'databses' at line 1
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| aa                 |
| mysql              |
| performance_schema |
| tangcl             |
| test               |
+--------------------+
6 rows in set (0.00 sec)

165的：

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| aa                 |
| mysql              |
| performance_schema |
| tangcl             |
| test               |
+--------------------+
6 rows in set (0.00 sec)