一、需求说明

最近一直在学习mysql-mmm,想以后这个架构也能用在我们公司的业务上,我们公司的业务是单机多实例部署,所以也想把mysql-mmm部署成这样,功夫不负有心人,我成功了,和大家分享一下:
二、环境说明

集群 角色 主机名 IP Mysql Port Server ID VIP Writer VIP READER
navy2 Agent db1 172.28.26.101 3307 11 172.28.26.107
navy3
Agent
db2 172.28.26.102 3308 1
172.28.26.110
navy2
Agent
db1
172.28.26.101
3307 22
172.28.26.108
navy3
Agent
db2
172.28.26.102
3308 2
172.28.26.109
navy2/navy3 Monitor
Monitor
172.28.26.103
PS:db1和db2上分别有两个库navy2和navy3,互为主主,172.28.26.107是navy2的写入虚拟IP,172.28.26.108是navy2的读虚拟IP,172.28.26.109是navy2的写入虚拟IP,172.28.26.110是navy3的读虚拟IP。
三、部署
1、mysql和mysql-mmm的安装及mysql的主从配置请查看前面的博文:http://navyaijm.blog.51cto.com/4647068/1230674,这里只介绍mmm多实例配置。
2、db1上:
vi /etc/mysql-mmm/mmm_common_navy2.conf(navy2的配置文件)
 
 vi /etc/mysql-mmm/mmm_common_navy2.conf
active_master_role writer
<host default>
cluster_interface eth1
agent_port 9912
mysql_port 3307
pid_path /var/run/mysql-mmm/mmm_agentd_navy2.pid
bin_path /usr/libexec/mysql-mmm/
replication_user slave
replication_password 123456
agent_user mmm_agent
agent_password 123456
</host>
<host db1>
ip 172.28.26.101
mysql_port 3307
mode master
peer db2
</host>
<host db2>
ip 172.28.26.102
mysql_port 3307
mode master
peer db1
</host>
<role writer>
hosts db1, db2
ips 172.28.26.107
mode exclusive
</role>
<role reader>
hosts db1, db2
ips 172.28.26.108
mode balanced
</role>

vi /etc/mysql-mmm/mmm_common_navy3.conf(navy的配置文件)

 active_master_role      writer
<host default>
cluster_interface eth1
agent_port 9913
mysql_port 3308
pid_path /var/run/mysql-mmm/mmm_agentd_navy3.pid
bin_path /usr/libexec/mysql-mmm/
replication_user slave
replication_password 123456
agent_user mmm_agent
agent_password 123456
</host>
<host db1>
ip 172.28.26.101
mysql_port 3308
mode master
peer db2
</host>
<host db2>
ip 172.28.26.102
mysql_port 3308
mode master
peer db1
</host>
<role writer>
hosts db1, db2
ips 172.28.26.109
mode exclusive
</role>
<role reader>
hosts db1, db2
ips 172.28.26.110
mode balanced
</role>

vi /etc/mysql-mmm/mmm_agent_navy2.conf(navy2的agent配置文件)

 include mmm_common_navy2.conf
this db1

vi /etc/mysql-mmm/mmm_agent_navy3.conf(navy3的agent配置文件)

 include mmm_common_navy3.conf
this db1

vi /etc/init.d/mysql-mmm-agent-navy2(navy2的agent启动脚本)

 #!/bin/sh
# chkconfig: - 64 36
# description: MMM Agent.
# processname: mmm_agentd
# config: /etc/mysql-mmm/mmm_agent.conf
# pidfile: /var/run/mysql-mmm/mmm_agentd.pid
# Source function library and defaults file.
. /etc/rc.d/init.d/functions
. /etc/default/mysql-mmm-agent
# Cluster name (it can be empty for default cases)
CLUSTER='navy2'
LOCKFILE='/var/lock/subsys/mysql-mmm-agent_navy2'
prog='MMM Agent Daemon'
#-----------------------------------------------------------------------
# Paths
if [ "$CLUSTER" != "" ]; then
MMMD_AGENT_BIN="/usr/sbin/mmm_agentd @$CLUSTER"
MMMD_AGENT_PIDFILE="/var/run/mysql-mmm/mmm_agentd_$CLUSTER.pid"
else
MMMD_AGENT_BIN="/usr/sbin/mmm_agentd"
MMMD_AGENT_PIDFILE="/var/run/mysql-mmm/mmm_agentd.pid"
fi
start() {
if [ "${ENABLED}" != "1" ]; then
echo "$prog is disabled!"
exit 1
fi
echo -n "Starting $prog: "
if [ -s $MMMD_AGENT_PIDFILE ] && kill -0 `cat $MMMD_AGENT_PIDFILE` 2> /dev/null; then
echo " already running."
exit 0
fi
daemon $MMMD_AGENT_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch $LOCKFILE
return $RETVAL
}
stop() {
# Stop daemon.
echo -n "Stopping $prog: "
killproc -p $MMMD_AGENT_PIDFILE $MMMD_AGENT_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $LOCKFILE
return $RETVAL
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $MMMD_AGENT_PIDFILE $MMMD_AGENT_BIN
RETVAL=$?
;;
restart|reload)
stop
start
;;
condrestart)
if [ -f $LOCKFILE ]; then
stop
start
fi
;;
*)
echo "Usage: $0 {start|stop|restart|condrestart|status}"
;;
esac
exit $RETVAL

赋予执行权限:

 chmod +x /etc/init.d/mysql-mmm-agent-navy2

vi /etc/init.d/mysql-mmm-agent-navy3(navy3的agent启动脚本)

 #!/bin/sh
# chkconfig: - 64 36
# description: MMM Agent.
# processname: mmm_agentd
# config: /etc/mysql-mmm/mmm_agent.conf
# pidfile: /var/run/mysql-mmm/mmm_agentd.pid
# Source function library and defaults file.
. /etc/rc.d/init.d/functions
. /etc/default/mysql-mmm-agent
## Paths
#MMMD_AGENT_BIN="/usr/sbin/mmm_agentd"
#MMMD_AGENT_PIDFILE="/var/run/mysql-mmm/mmm_agentd.pid"
#LOCKFILE='/var/lock/subsys/mysql-mmm-agent'
#prog='MMM Agent Daemon'
# Cluster name (it can be empty for default cases)
CLUSTER='navy3'
LOCKFILE='/var/lock/subsys/mysql-mmm-agent_navy3'
prog='MMM Agent Daemon'
#-----------------------------------------------------------------------
# Paths
if [ "$CLUSTER" != "" ]; then
MMMD_AGENT_BIN="/usr/sbin/mmm_agentd @$CLUSTER"
MMMD_AGENT_PIDFILE="/var/run/mysql-mmm/mmm_agentd_$CLUSTER.pid"
else
MMMD_AGENT_BIN="/usr/sbin/mmm_agentd"
MMMD_AGENT_PIDFILE="/var/run/mysql-mmm/mmm_agentd.pid"
fi
start() {
if [ "${ENABLED}" != "1" ]; then
echo "$prog is disabled!"
exit 1
fi
echo -n "Starting $prog: "
if [ -s $MMMD_AGENT_PIDFILE ] && kill -0 `cat $MMMD_AGENT_PIDFILE` 2> /dev/null; then
echo " already running."
exit 0
fi
daemon $MMMD_AGENT_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch $LOCKFILE
return $RETVAL
}
stop() {
# Stop daemon.
echo -n "Stopping $prog: "
killproc -p $MMMD_AGENT_PIDFILE $MMMD_AGENT_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $LOCKFILE
return $RETVAL
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $MMMD_AGENT_PIDFILE $MMMD_AGENT_BIN
RETVAL=$?
;;
restart|reload)
stop
start
;;
condrestart)
if [ -f $LOCKFILE ]; then
stop
start
fi
;;
*)
echo "Usage: $0 {start|stop|restart|condrestart|status}"
;;
esac
exit $RETVAL

赋予可执行权限:

 chmod +x /etc/init.d/mysql-mmm-agent-navy3

启动服务:

 /etc/init.d/mysql-mmm-agent-navy2 start
/etc/init.d/mysql-mmm-agent-navy3 start
3、db2上:
复制db1上的文件到相应的目录下:
 
 scp /etc/mysql-mmm/mmm_common_navy2.conf 172.28.26.102:/etc/mysql-mmm/
scp /etc/mysql-mmm/mmm_common_navy3.conf 172.28.26.102:/etc/mysql-mmm/
scp /etc/mysql-mmm/mmm_agent_navy2.conf 172.28.26.102:/etc/mysql-mmm/
scp /etc/mysql-mmm/mmm_agent_navy3.conf 172.28.26.102:/etc/mysql-mmm/
scp /etc/init.d/mysql-mmm-agent-navy2 172.28.26.102:/etc/init.d/
scp /etc/init.d/mysql-mmm-agent-navy3 172.28.26.102:/etc/init.d/

修改agent配置文件:

 sed -i 's/this db1/this db2/' /etc/mysql-mmm/mmm_agent_navy2.conf
sed -i 's/this db1/this db2/' /etc/mysql-mmm/mmm_agent_navy3.conf

赋予可执行权限

 chmod +x /etc/init.d/mysql-mmm-agent-navy2
chmod +x /etc/init.d/mysql-mmm-agent-navy3

启动服务:

 /etc/init.d/mysql-mmm-agent-navy2 start
/etc/init.d/mysql-mmm-agent-navy3 start
4、monitor上
复制db1上的配置文件:
 scp /etc/mysql-mmm/mmm_common_navy2.conf 172.28.26.103:/etc/mysql-mmm/
scp /etc/mysql-mmm/mmm_common_navy3.conf 172.28.26.103:/etc/mysql-mmm/

vi /etc/mysql-mmm/mmm_mon_navy2.conf

 include mmm_common_navy2.conf
<monitor>
ip 127.0.0.1
port 9992
pid_path /var/run/mysql-mmm/mmm_mond_navy2.pid
bin_path /usr/libexec/mysql-mmm
status_path /var/lib/mysql-mmm/mmm_mond_navy2.status
ping_ips 172.28.26.101,172.28.26.102
auto_set_online 10
# wait_for_other_master 2
# The kill_host_bin does not exist by default, though the monitor will
# throw a warning about it missing. See the section 5.10 "Kill Host
# Functionality" in the PDF documentation.
#
# kill_host_bin /usr/libexec/mysql-mmm/monitor/kill_host
#
</monitor>
<host default>
monitor_user mmm_monitor
monitor_password 123456
</host>
debug 0

vi /etc/mysql-mmm/mmm_mon_navy3.conf

 include mmm_common_navy3.conf
<monitor>
ip 127.0.0.1
port 9993
pid_path /var/run/mysql-mmm/mmm_mond_navy3.pid
bin_path /usr/libexec/mysql-mmm
status_path /var/lib/mysql-mmm/mmm_mond_navy3.status
ping_ips 172.28.26.101,172.28.26.102
auto_set_online 10
# wait_for_other_master 2
# The kill_host_bin does not exist by default, though the monitor will
# throw a warning about it missing. See the section 5.10 "Kill Host
# Functionality" in the PDF documentation.
#
# kill_host_bin /usr/libexec/mysql-mmm/monitor/kill_host
#
</monitor>
<host default>
monitor_user mmm_monitor
monitor_password 123456
</host>
debug 0

vi /etc/mysql-mmm/mmm_mon_log_navy2.conf

 #log4perl.logger = FATAL, MMMLog, MailFatal
log4perl.logger = FATAL, MMMLog
log4perl.appender.MMMLog = Log::Log4perl::Appender::File
log4perl.appender.MMMLog.Threshold = INFO
log4perl.appender.MMMLog.filename = /var/log/mysql-mmm/mmm_mond_navy2.log
log4perl.appender.MMMLog.recreate = 1
log4perl.appender.MMMLog.layout = PatternLayout
log4perl.appender.MMMLog.layout.ConversionPattern = %d %5p %m%n
#log4perl.appender.MailFatal = Log::Dispatch::Email::MailSender
#log4perl.appender.MailFatal.Threshold = FATAL
#log4perl.appender.MailFatal.from = mmm@example.com
#log4perl.appender.MailFatal.to = root
#log4perl.appender.MailFatal.buffered = 0
#log4perl.appender.MailFatal.subject = FATAL error in mysql-mmm-monitor
#log4perl.appender.MailFatal.layout = PatternLayout
#log4perl.appender.MailFatal.layout.ConversionPattern = %d %m%n

vi /etc/mysql-mmm/mmm_mon_log_navy3.conf

 #log4perl.logger = FATAL, MMMLog, MailFatal
log4perl.logger = FATAL, MMMLog
log4perl.appender.MMMLog = Log::Log4perl::Appender::File
log4perl.appender.MMMLog.Threshold = INFO
log4perl.appender.MMMLog.filename = /var/log/mysql-mmm/mmm_mond_navy3.log
log4perl.appender.MMMLog.recreate = 1
log4perl.appender.MMMLog.layout = PatternLayout
log4perl.appender.MMMLog.layout.ConversionPattern = %d %5p %m%n
#log4perl.appender.MailFatal = Log::Dispatch::Email::MailSender
#log4perl.appender.MailFatal.Threshold = FATAL
#log4perl.appender.MailFatal.from = mmm@example.com
#log4perl.appender.MailFatal.to = root
#log4perl.appender.MailFatal.buffered = 0
#log4perl.appender.MailFatal.subject = FATAL error in mysql-mmm-monitor
#log4perl.appender.MailFatal.layout = PatternLayout
#log4perl.appender.MailFatal.layout.ConversionPattern = %d %m%n

vi /etc/init.d/mysql-mmm-monitor-navy2

 #!/bin/sh
#
# mysql-mmm-monitor This shell script takes care of starting and stopping
# the mmm monitoring daemon.
#
# chkconfig: - 64 36
# description: MMM Monitor.
# processname: mmm_mond
# config: /etc/mysql-mmm/mmm_mon.conf
# pidfile: /var/run/mysql-mmm/mmm_mond.pid
# Source function library and defaults file.
. /etc/rc.d/init.d/functions
. /etc/default/mysql-mmm-monitor
# Cluster name (it can be empty for default cases)
CLUSTER='navy2'
LOCKFILE='/var/lock/subsys/mysql-mmm-monitor_navy2'
prog='MMM Monitor Daemon'
if [ "$CLUSTER" != "" ]; then
MMMD_MON_BIN="/usr/sbin/mmm_mond @$CLUSTER"
MMMD_MON_PIDFILE="/var/run/mysql-mmm/mmm_mond-$CLUSTER.pid"
else
MMMD_MON_BIN="/usr/sbin/mmm_mond"
MMMD_MON_PIDFILE="/var/run/mysql-mmm/mmm_mond.pid"
fi
start() {
if [ "${ENABLED}" != "1" ]; then
echo "$prog is disabled!"
exit 1
fi
echo -n "Starting $prog: "
if [ -s $MMMD_MON_PIDFILE ] && kill -0 `cat $MMMD_MON_PIDFILE` 2> /dev/null; then
echo " already running."
exit 0
fi
daemon $MMMD_MON_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch $LOCKFILE
return $RETVAL
}
stop() {
# Stop daemon.
echo -n "Stopping $prog: "
killproc -p $MMMD_MON_PIDFILE $MMMD_MON_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $LOCKFILE
return $RETVAL
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $MMMD_MON_PIDFILE $MMMD_MON_BIN
RETVAL=$?
;;
restart|reload)
stop
start
;;
condrestart)
if [ -f $LOCKFILE ]; then
stop
start
fi
;;
*)
echo "Usage: $0 {start|stop|restart|condrestart|status}"
;;
esac
exit $RETVAL

vi /etc/init.d/mysql-mmm-monitor-navy3

 #!/bin/sh
#
# mysql-mmm-monitor This shell script takes care of starting and stopping
# the mmm monitoring daemon.
#
# chkconfig: - 64 36
# description: MMM Monitor.
# processname: mmm_mond
# config: /etc/mysql-mmm/mmm_mon.conf
# pidfile: /var/run/mysql-mmm/mmm_mond.pid
# Source function library and defaults file.
. /etc/rc.d/init.d/functions
. /etc/default/mysql-mmm-monitor
# Cluster name (it can be empty for default cases)
CLUSTER='navy3'
LOCKFILE='/var/lock/subsys/mysql-mmm-monitor_navy3'
prog='MMM Monitor Daemon'
if [ "$CLUSTER" != "" ]; then
MMMD_MON_BIN="/usr/sbin/mmm_mond @$CLUSTER"
MMMD_MON_PIDFILE="/var/run/mysql-mmm/mmm_mond-$CLUSTER.pid"
else
MMMD_MON_BIN="/usr/sbin/mmm_mond"
MMMD_MON_PIDFILE="/var/run/mysql-mmm/mmm_mond.pid"
fi
start() {
if [ "${ENABLED}" != "1" ]; then
echo "$prog is disabled!"
exit 1
fi
echo -n "Starting $prog: "
if [ -s $MMMD_MON_PIDFILE ] && kill -0 `cat $MMMD_MON_PIDFILE` 2> /dev/null; then
echo " already running."
exit 0
fi
daemon $MMMD_MON_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch $LOCKFILE
return $RETVAL
}
stop() {
# Stop daemon.
echo -n "Stopping $prog: "
killproc -p $MMMD_MON_PIDFILE $MMMD_MON_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $LOCKFILE
return $RETVAL
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $MMMD_MON_PIDFILE $MMMD_MON_BIN
RETVAL=$?
;;
restart|reload)
stop
start
;;
condrestart)
if [ -f $LOCKFILE ]; then
stop
start
fi
;;
*)
echo "Usage: $0 {start|stop|restart|condrestart|status}"
;;
esac
exit $RETVAL

赋予可执行权限:

 chmod +x /etc/init.d/mysql-mmm-monitor-navy2
chmod +x /etc/init.d/mysql-mmm-monitor-navy3

启动监控服务:

/etc/init.d/mysql-mmm-monitor-navy2 start
/etc/init.d/mysql-mmm-monitor-navy3 start

结果

 [root@monitor ~]# mmm_control  show
db1(172.28.26.101) master/ONLINE. Roles: writer(172.28.26.104)
db2(172.28.26.102) master/ONLINE. Roles:
db3(172.28.26.188) slave/ONLINE. Roles: reader(172.28.26.105)
db4(172.28.26.189) slave/ONLINE. Roles: reader(172.28.26.106)
[root@monitor ~]# mmm_control @navy2 show
db1(172.28.26.101) master/ONLINE. Roles: writer(172.28.26.107)
db2(172.28.26.102) master/ONLINE. Roles: reader(172.28.26.108)
[root@monitor ~]# mmm_control @navy3 show
db1(172.28.26.101) master/ONLINE. Roles: writer(172.28.26.109)
db2(172.28.26.102) master/ONLINE. Roles: reader(172.28.26.110)
[root@monitor ~]#

[转]mysql-mmm集群(多实例)的更多相关文章

  1. 多图文,详细介绍mysql各个集群方案

    目录 多图文,详细介绍mysql各个集群方案 一,mysql原厂出品 二,mysql第三方优化 三,依托硬件配合 四,其它 多图文,详细介绍mysql各个集群方案 集群的好处 高可用性:故障检测及迁移 ...

  2. MySQL分布式集群之MyCAT(转)

    原文地址:http://blog.itpub.net/29510932/viewspace-1664499/ 隔了好久,才想起来更新博客,最近倒腾的数据库从Oracle换成了MySQL,研究了一段时间 ...

  3. MySQL Cluster 集群

    本文转载 http://www.cnblogs.com/gomysql/p/3664783.html MySQL Cluster是一个基于NDB Cluster存储引擎的完整的分布式数据库系统.不仅仅 ...

  4. 基于keepalived搭建MySQL热机集群

    背景 MySQL的高可用方案一般有如下几种: keepalived+双主,MHA,MMM,Heartbeat+DRBD,PXC,Galera Cluster 比较常用的是keepalived+双主,M ...

  5. MySQL分布式集群之MyCAT(一)简介【转】

    隔了好久,才想起来更新博客,最近倒腾的数据库从Oracle换成了MySQL,研究了一段时间,感觉社区版的MySQL在各个方面都逊色于Oracle,Oracle真的好方便!好了,不废话,这次准备记录一些 ...

  6. mysql各种集群的优缺点

    mysql各种集群的优缺点 1.主从架构:只是有数据备份的功能: 2.主主互备+keepalived:实现数据备份加高可用: 3.主主互备,主主下面分别挂个从: 4.A和B主主互备,把从库都挂到B下, ...

  7. 《综合》MMM集群

    <综合>MMM集群 部署集群基础环境 MySQL-MMM架构部署 MySQL-MMM架构使用 1 部署集群基础环境 1.1 问题 本案例要求为MySQL集群准备基础环境,完成以下任务操作: ...

  8. Shell脚本实现---Swarm集群部署实例(Swarm Cluster)

    Shell脚本实现---Swarm集群部署实例(Swarm Cluster) 一.机器环境(均是centos7.8) IP hostname 角色 192.168.10.200 manager-swa ...

  9. MySQL数据库集群进行正确配置步骤

    MySQL数据库集群进行正确配置步骤 2010-06-09 10:47 arrowcat 博客园 字号:T | T 我们今天是要和大家一起分享的是对MySQL数据库集群进行正确配置,我前两天在相关网站 ...

  10. Facebook揭密:如何让MySQL数据库集群自主运行

    Facebook运行着全球最大的MySQL数据库集群,该集群分布在两个大洲上的多个数据中心中数以千计的服务器上.让人不解的是,Facebook只动用了一个很小的团队来管理这个庞大的MySQL数据库集群 ...

随机推荐

  1. 27. Remove Element(双指针)

      Given an array nums and a value val, remove all instances of that value in-place and return the ne ...

  2. 服务器返回的json数据中含有null的处理方法

    个人博客:http://guohuaden.com/2017/03/06/json-dataNull/因为有遇到过类似情况,所以就想到了一些解决方法,并且实践了一下,这里简单的做个记录. 注:有看到不 ...

  3. linux服务器上nginx日志访问量统计命令

    linux服务器上nginx日志访问量统计命令 日志文件所在地方:/var/log/nginx/access_iqueendress.com.log/var/log/nginx/access_m.iq ...

  4. windows下nodejs express安装及入门网站,视频资料,开源项目介绍

    windows下nodejs express安装及入门网站,视频资料,开源项目介绍,pm2,supervisor,npm,Pomelo,Grunt安装使用注意事项等总结 第一步:下载安装文件下载地址: ...

  5. [转载]论asp.net out、ref、return

      论asp.net out.ref.return ref(引用类型) ref引用类型进出都必须赋值,赋值后会改变类型原来的指针.   out(值类型) out值类型进可以不赋值,出必须赋值.   r ...

  6. Python Web学习笔记之socket编程

    Python 提供了两个基本的 socket 模块. 第一个是 Socket,它提供了标准的 BSD Sockets API. 第二个是 SocketServer, 它提供了服务器中心类,可以简化网络 ...

  7. bzoj1648 / P2853 [USACO06DEC]牛的野餐Cow Picnic

    P2853 [USACO06DEC]牛的野餐Cow Picnic 你愿意的话,可以写dj. 然鹅,对一个缺时间的退役选手来说,暴力模拟是一个不错的选择. 让每个奶牛都把图走一遍,显然那些被每个奶牛都走 ...

  8. 02:Django进阶篇

    目录:Django其他篇 01:Django基础篇 02:Django进阶篇 03:Django数据库操作--->Model 04: Form 验证用户数据 & 生成html 05:Mo ...

  9. 20145220韩旭飞《网络对抗》Exp7 网络欺诈技术防范

    20145220韩旭飞<网络对抗>Exp7 网络欺诈技术防范 应用SET工具建立冒名网站 要让冒名网站在别的主机上也能看到,需要开启本机的Apache服务,并且要将Apache服务的默认端 ...

  10. 20145208 蔡野《网络对抗》shellcode注入&Return-to-libc攻击深入

    20145208 蔡野<网络对抗>shellcode注入&Return-to-libc攻击深入 Shellcode注入 shellcode的获取代码 我使用了许心远同学博客中的代码 ...