[转]mysql-mmm集群（多实例）
一、需求说明
最近一直在学习mysql-mmm，想以后这个架构也能用在我们公司的业务上，我们公司的业务是单机多实例部署，所以也想把mysql-mmm部署成这样，功夫不负有心人，我成功了，和大家分享一下：
二、环境说明
| 集群 | 角色 | 主机名 | IP | Mysql Port | Server ID | VIP Writer | VIP READER |
| navy2 | Agent | db1 | 172.28.26.101 | 3307 | 11 | 172.28.26.107 | |
| navy3 |
Agent
|
db2 | 172.28.26.102 | 3308 | 1 | |
172.28.26.110
|
| navy2 |
Agent
|
db1 |
172.28.26.101
|
3307 | 22 | |
172.28.26.108
|
| navy3 |
Agent
|
db2 |
172.28.26.102
|
3308 | 2 |
172.28.26.109
|
|
| navy2/navy3 | Monitor |
Monitor
|
172.28.26.103
|
| | | |
PS：db1和db2上分别有两个库navy2和navy3，互为主主，172.28.26.107是navy2的写入虚拟IP，172.28.26.108是navy2的读虚拟IP，172.28.26.109是navy2的写入虚拟IP，172.28.26.110是navy3的读虚拟IP。
三、部署
1、mysql和mysql-mmm的安装及mysql的主从配置请查看前面的博文：http://navyaijm.blog.51cto.com/4647068/1230674，这里只介绍mmm多实例配置。
2、db1上：
vi /etc/mysql-mmm/mmm_common_navy2.conf（navy2的配置文件）
vi /etc/mysql-mmm/mmm_common_navy2.conf
active_master_role writer
<host default>
cluster_interface eth1
agent_port 9912
mysql_port 3307
pid_path /var/run/mysql-mmm/mmm_agentd_navy2.pid
bin_path /usr/libexec/mysql-mmm/
replication_user slave
replication_password 123456
agent_user mmm_agent
agent_password 123456
</host>
<host db1>
ip 172.28.26.101
mysql_port 3307
mode master
peer db2
</host>
<host db2>
ip 172.28.26.102
mysql_port 3307
mode master
peer db1
</host>
<role writer>
hosts db1, db2
ips 172.28.26.107
mode exclusive
</role>
<role reader>
hosts db1, db2
ips 172.28.26.108
mode balanced
</role>
vi /etc/mysql-mmm/mmm_common_navy3.conf（navy的配置文件）
active_master_role writer
<host default>
cluster_interface eth1
agent_port 9913
mysql_port 3308
pid_path /var/run/mysql-mmm/mmm_agentd_navy3.pid
bin_path /usr/libexec/mysql-mmm/
replication_user slave
replication_password 123456
agent_user mmm_agent
agent_password 123456
</host>
<host db1>
ip 172.28.26.101
mysql_port 3308
mode master
peer db2
</host>
<host db2>
ip 172.28.26.102
mysql_port 3308
mode master
peer db1
</host>
<role writer>
hosts db1, db2
ips 172.28.26.109
mode exclusive
</role>
<role reader>
hosts db1, db2
ips 172.28.26.110
mode balanced
</role>
vi /etc/mysql-mmm/mmm_agent_navy2.conf（navy2的agent配置文件）
include mmm_common_navy2.conf
this db1
vi /etc/mysql-mmm/mmm_agent_navy3.conf（navy3的agent配置文件）
include mmm_common_navy3.conf
this db1
vi /etc/init.d/mysql-mmm-agent-navy2（navy2的agent启动脚本）
#!/bin/sh
# chkconfig: - 64 36
# description: MMM Agent.
# processname: mmm_agentd
# config: /etc/mysql-mmm/mmm_agent.conf
# pidfile: /var/run/mysql-mmm/mmm_agentd.pid
# Source function library and defaults file.
. /etc/rc.d/init.d/functions
. /etc/default/mysql-mmm-agent
# Cluster name (it can be empty for default cases)
CLUSTER='navy2'
LOCKFILE='/var/lock/subsys/mysql-mmm-agent_navy2'
prog='MMM Agent Daemon'
#-----------------------------------------------------------------------
# Paths
if [ "$CLUSTER" != "" ]; then
MMMD_AGENT_BIN="/usr/sbin/mmm_agentd @$CLUSTER"
MMMD_AGENT_PIDFILE="/var/run/mysql-mmm/mmm_agentd_$CLUSTER.pid"
else
MMMD_AGENT_BIN="/usr/sbin/mmm_agentd"
MMMD_AGENT_PIDFILE="/var/run/mysql-mmm/mmm_agentd.pid"
fi
start() {
if [ "${ENABLED}" != "1" ]; then
echo "$prog is disabled!"
exit 1
fi
echo -n "Starting $prog: "
if [ -s $MMMD_AGENT_PIDFILE ] && kill -0 `cat $MMMD_AGENT_PIDFILE` 2> /dev/null; then
echo " already running."
exit 0
fi
daemon $MMMD_AGENT_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch $LOCKFILE
return $RETVAL
}
stop() {
# Stop daemon.
echo -n "Stopping $prog: "
killproc -p $MMMD_AGENT_PIDFILE $MMMD_AGENT_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $LOCKFILE
return $RETVAL
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $MMMD_AGENT_PIDFILE $MMMD_AGENT_BIN
RETVAL=$?
;;
restart|reload)
stop
start
;;
condrestart)
if [ -f $LOCKFILE ]; then
stop
start
fi
;;
*)
echo "Usage: $0 {start|stop|restart|condrestart|status}"
;;
esac
exit $RETVAL
赋予执行权限：
chmod +x /etc/init.d/mysql-mmm-agent-navy2
vi /etc/init.d/mysql-mmm-agent-navy3（navy3的agent启动脚本）
#!/bin/sh
# chkconfig: - 64 36
# description: MMM Agent.
# processname: mmm_agentd
# config: /etc/mysql-mmm/mmm_agent.conf
# pidfile: /var/run/mysql-mmm/mmm_agentd.pid
# Source function library and defaults file.
. /etc/rc.d/init.d/functions
. /etc/default/mysql-mmm-agent
## Paths
#MMMD_AGENT_BIN="/usr/sbin/mmm_agentd"
#MMMD_AGENT_PIDFILE="/var/run/mysql-mmm/mmm_agentd.pid"
#LOCKFILE='/var/lock/subsys/mysql-mmm-agent'
#prog='MMM Agent Daemon'
# Cluster name (it can be empty for default cases)
CLUSTER='navy3'
LOCKFILE='/var/lock/subsys/mysql-mmm-agent_navy3'
prog='MMM Agent Daemon'
#-----------------------------------------------------------------------
# Paths
if [ "$CLUSTER" != "" ]; then
MMMD_AGENT_BIN="/usr/sbin/mmm_agentd @$CLUSTER"
MMMD_AGENT_PIDFILE="/var/run/mysql-mmm/mmm_agentd_$CLUSTER.pid"
else
MMMD_AGENT_BIN="/usr/sbin/mmm_agentd"
MMMD_AGENT_PIDFILE="/var/run/mysql-mmm/mmm_agentd.pid"
fi
start() {
if [ "${ENABLED}" != "1" ]; then
echo "$prog is disabled!"
exit 1
fi
echo -n "Starting $prog: "
if [ -s $MMMD_AGENT_PIDFILE ] && kill -0 `cat $MMMD_AGENT_PIDFILE` 2> /dev/null; then
echo " already running."
exit 0
fi
daemon $MMMD_AGENT_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch $LOCKFILE
return $RETVAL
}
stop() {
# Stop daemon.
echo -n "Stopping $prog: "
killproc -p $MMMD_AGENT_PIDFILE $MMMD_AGENT_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $LOCKFILE
return $RETVAL
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $MMMD_AGENT_PIDFILE $MMMD_AGENT_BIN
RETVAL=$?
;;
restart|reload)
stop
start
;;
condrestart)
if [ -f $LOCKFILE ]; then
stop
start
fi
;;
*)
echo "Usage: $0 {start|stop|restart|condrestart|status}"
;;
esac
exit $RETVAL
赋予可执行权限：
chmod +x /etc/init.d/mysql-mmm-agent-navy3
启动服务：
/etc/init.d/mysql-mmm-agent-navy2 start
/etc/init.d/mysql-mmm-agent-navy3 start
3、db2上：
复制db1上的文件到相应的目录下：
scp /etc/mysql-mmm/mmm_common_navy2.conf 172.28.26.102:/etc/mysql-mmm/
scp /etc/mysql-mmm/mmm_common_navy3.conf 172.28.26.102:/etc/mysql-mmm/
scp /etc/mysql-mmm/mmm_agent_navy2.conf 172.28.26.102:/etc/mysql-mmm/
scp /etc/mysql-mmm/mmm_agent_navy3.conf 172.28.26.102:/etc/mysql-mmm/
scp /etc/init.d/mysql-mmm-agent-navy2 172.28.26.102:/etc/init.d/
scp /etc/init.d/mysql-mmm-agent-navy3 172.28.26.102:/etc/init.d/
修改agent配置文件：
sed -i 's/this db1/this db2/' /etc/mysql-mmm/mmm_agent_navy2.conf
sed -i 's/this db1/this db2/' /etc/mysql-mmm/mmm_agent_navy3.conf
赋予可执行权限
chmod +x /etc/init.d/mysql-mmm-agent-navy2
chmod +x /etc/init.d/mysql-mmm-agent-navy3
启动服务：
/etc/init.d/mysql-mmm-agent-navy2 start
/etc/init.d/mysql-mmm-agent-navy3 start
4、monitor上
复制db1上的配置文件：
scp /etc/mysql-mmm/mmm_common_navy2.conf 172.28.26.103:/etc/mysql-mmm/
scp /etc/mysql-mmm/mmm_common_navy3.conf 172.28.26.103:/etc/mysql-mmm/
vi /etc/mysql-mmm/mmm_mon_navy2.conf
include mmm_common_navy2.conf
<monitor>
ip 127.0.0.1
port 9992
pid_path /var/run/mysql-mmm/mmm_mond_navy2.pid
bin_path /usr/libexec/mysql-mmm
status_path /var/lib/mysql-mmm/mmm_mond_navy2.status
ping_ips 172.28.26.101,172.28.26.102
auto_set_online 10
# wait_for_other_master 2
# The kill_host_bin does not exist by default, though the monitor will
# throw a warning about it missing. See the section 5.10 "Kill Host
# Functionality" in the PDF documentation.
#
# kill_host_bin /usr/libexec/mysql-mmm/monitor/kill_host
#
</monitor>
<host default>
monitor_user mmm_monitor
monitor_password 123456
</host>
debug 0
vi /etc/mysql-mmm/mmm_mon_navy3.conf
include mmm_common_navy3.conf
<monitor>
ip 127.0.0.1
port 9993
pid_path /var/run/mysql-mmm/mmm_mond_navy3.pid
bin_path /usr/libexec/mysql-mmm
status_path /var/lib/mysql-mmm/mmm_mond_navy3.status
ping_ips 172.28.26.101,172.28.26.102
auto_set_online 10
# wait_for_other_master 2
# The kill_host_bin does not exist by default, though the monitor will
# throw a warning about it missing. See the section 5.10 "Kill Host
# Functionality" in the PDF documentation.
#
# kill_host_bin /usr/libexec/mysql-mmm/monitor/kill_host
#
</monitor>
<host default>
monitor_user mmm_monitor
monitor_password 123456
</host>
debug 0
vi /etc/mysql-mmm/mmm_mon_log_navy2.conf
#log4perl.logger = FATAL, MMMLog, MailFatal
log4perl.logger = FATAL, MMMLog
log4perl.appender.MMMLog = Log::Log4perl::Appender::File
log4perl.appender.MMMLog.Threshold = INFO
log4perl.appender.MMMLog.filename = /var/log/mysql-mmm/mmm_mond_navy2.log
log4perl.appender.MMMLog.recreate = 1
log4perl.appender.MMMLog.layout = PatternLayout
log4perl.appender.MMMLog.layout.ConversionPattern = %d %5p %m%n
#log4perl.appender.MailFatal = Log::Dispatch::Email::MailSender
#log4perl.appender.MailFatal.Threshold = FATAL
#log4perl.appender.MailFatal.from = mmm@example.com
#log4perl.appender.MailFatal.to = root
#log4perl.appender.MailFatal.buffered = 0
#log4perl.appender.MailFatal.subject = FATAL error in mysql-mmm-monitor
#log4perl.appender.MailFatal.layout = PatternLayout
#log4perl.appender.MailFatal.layout.ConversionPattern = %d %m%n
vi /etc/mysql-mmm/mmm_mon_log_navy3.conf
#log4perl.logger = FATAL, MMMLog, MailFatal
log4perl.logger = FATAL, MMMLog
log4perl.appender.MMMLog = Log::Log4perl::Appender::File
log4perl.appender.MMMLog.Threshold = INFO
log4perl.appender.MMMLog.filename = /var/log/mysql-mmm/mmm_mond_navy3.log
log4perl.appender.MMMLog.recreate = 1
log4perl.appender.MMMLog.layout = PatternLayout
log4perl.appender.MMMLog.layout.ConversionPattern = %d %5p %m%n
#log4perl.appender.MailFatal = Log::Dispatch::Email::MailSender
#log4perl.appender.MailFatal.Threshold = FATAL
#log4perl.appender.MailFatal.from = mmm@example.com
#log4perl.appender.MailFatal.to = root
#log4perl.appender.MailFatal.buffered = 0
#log4perl.appender.MailFatal.subject = FATAL error in mysql-mmm-monitor
#log4perl.appender.MailFatal.layout = PatternLayout
#log4perl.appender.MailFatal.layout.ConversionPattern = %d %m%n
vi /etc/init.d/mysql-mmm-monitor-navy2
#!/bin/sh
#
# mysql-mmm-monitor This shell script takes care of starting and stopping
# the mmm monitoring daemon.
#
# chkconfig: - 64 36
# description: MMM Monitor.
# processname: mmm_mond
# config: /etc/mysql-mmm/mmm_mon.conf
# pidfile: /var/run/mysql-mmm/mmm_mond.pid
# Source function library and defaults file.
. /etc/rc.d/init.d/functions
. /etc/default/mysql-mmm-monitor
# Cluster name (it can be empty for default cases)
CLUSTER='navy2'
LOCKFILE='/var/lock/subsys/mysql-mmm-monitor_navy2'
prog='MMM Monitor Daemon'
if [ "$CLUSTER" != "" ]; then
MMMD_MON_BIN="/usr/sbin/mmm_mond @$CLUSTER"
MMMD_MON_PIDFILE="/var/run/mysql-mmm/mmm_mond-$CLUSTER.pid"
else
MMMD_MON_BIN="/usr/sbin/mmm_mond"
MMMD_MON_PIDFILE="/var/run/mysql-mmm/mmm_mond.pid"
fi
start() {
if [ "${ENABLED}" != "1" ]; then
echo "$prog is disabled!"
exit 1
fi
echo -n "Starting $prog: "
if [ -s $MMMD_MON_PIDFILE ] && kill -0 `cat $MMMD_MON_PIDFILE` 2> /dev/null; then
echo " already running."
exit 0
fi
daemon $MMMD_MON_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch $LOCKFILE
return $RETVAL
}
stop() {
# Stop daemon.
echo -n "Stopping $prog: "
killproc -p $MMMD_MON_PIDFILE $MMMD_MON_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $LOCKFILE
return $RETVAL
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $MMMD_MON_PIDFILE $MMMD_MON_BIN
RETVAL=$?
;;
restart|reload)
stop
start
;;
condrestart)
if [ -f $LOCKFILE ]; then
stop
start
fi
;;
*)
echo "Usage: $0 {start|stop|restart|condrestart|status}"
;;
esac
exit $RETVAL
vi /etc/init.d/mysql-mmm-monitor-navy3
#!/bin/sh
#
# mysql-mmm-monitor This shell script takes care of starting and stopping
# the mmm monitoring daemon.
#
# chkconfig: - 64 36
# description: MMM Monitor.
# processname: mmm_mond
# config: /etc/mysql-mmm/mmm_mon.conf
# pidfile: /var/run/mysql-mmm/mmm_mond.pid
# Source function library and defaults file.
. /etc/rc.d/init.d/functions
. /etc/default/mysql-mmm-monitor
# Cluster name (it can be empty for default cases)
CLUSTER='navy3'
LOCKFILE='/var/lock/subsys/mysql-mmm-monitor_navy3'
prog='MMM Monitor Daemon'
if [ "$CLUSTER" != "" ]; then
MMMD_MON_BIN="/usr/sbin/mmm_mond @$CLUSTER"
MMMD_MON_PIDFILE="/var/run/mysql-mmm/mmm_mond-$CLUSTER.pid"
else
MMMD_MON_BIN="/usr/sbin/mmm_mond"
MMMD_MON_PIDFILE="/var/run/mysql-mmm/mmm_mond.pid"
fi
start() {
if [ "${ENABLED}" != "1" ]; then
echo "$prog is disabled!"
exit 1
fi
echo -n "Starting $prog: "
if [ -s $MMMD_MON_PIDFILE ] && kill -0 `cat $MMMD_MON_PIDFILE` 2> /dev/null; then
echo " already running."
exit 0
fi
daemon $MMMD_MON_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch $LOCKFILE
return $RETVAL
}
stop() {
# Stop daemon.
echo -n "Stopping $prog: "
killproc -p $MMMD_MON_PIDFILE $MMMD_MON_BIN
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $LOCKFILE
return $RETVAL
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $MMMD_MON_PIDFILE $MMMD_MON_BIN
RETVAL=$?
;;
restart|reload)
stop
start
;;
condrestart)
if [ -f $LOCKFILE ]; then
stop
start
fi
;;
*)
echo "Usage: $0 {start|stop|restart|condrestart|status}"
;;
esac
exit $RETVAL
赋予可执行权限：
chmod +x /etc/init.d/mysql-mmm-monitor-navy2
chmod +x /etc/init.d/mysql-mmm-monitor-navy3
启动监控服务：
/etc/init.d/mysql-mmm-monitor-navy2 start
/etc/init.d/mysql-mmm-monitor-navy3 start
结果
[root@monitor ~]# mmm_control show
db1(172.28.26.101) master/ONLINE. Roles: writer(172.28.26.104)
db2(172.28.26.102) master/ONLINE. Roles:
db3(172.28.26.188) slave/ONLINE. Roles: reader(172.28.26.105)
db4(172.28.26.189) slave/ONLINE. Roles: reader(172.28.26.106)
[root@monitor ~]# mmm_control @navy2 show
db1(172.28.26.101) master/ONLINE. Roles: writer(172.28.26.107)
db2(172.28.26.102) master/ONLINE. Roles: reader(172.28.26.108)
[root@monitor ~]# mmm_control @navy3 show
db1(172.28.26.101) master/ONLINE. Roles: writer(172.28.26.109)
db2(172.28.26.102) master/ONLINE. Roles: reader(172.28.26.110)
[root@monitor ~]#
[转]mysql-mmm集群（多实例）的更多相关文章
- 多图文，详细介绍mysql各个集群方案
目录 多图文,详细介绍mysql各个集群方案 一,mysql原厂出品 二,mysql第三方优化 三,依托硬件配合 四,其它 多图文,详细介绍mysql各个集群方案 集群的好处 高可用性:故障检测及迁移 ...
- MySQL分布式集群之MyCAT(转)
原文地址:http://blog.itpub.net/29510932/viewspace-1664499/ 隔了好久,才想起来更新博客,最近倒腾的数据库从Oracle换成了MySQL,研究了一段时间 ...
- MySQL Cluster 集群
本文转载 http://www.cnblogs.com/gomysql/p/3664783.html MySQL Cluster是一个基于NDB Cluster存储引擎的完整的分布式数据库系统.不仅仅 ...
- 基于keepalived搭建MySQL热机集群
背景 MySQL的高可用方案一般有如下几种: keepalived+双主,MHA,MMM,Heartbeat+DRBD,PXC,Galera Cluster 比较常用的是keepalived+双主,M ...
- MySQL分布式集群之MyCAT（一）简介【转】
隔了好久,才想起来更新博客,最近倒腾的数据库从Oracle换成了MySQL,研究了一段时间,感觉社区版的MySQL在各个方面都逊色于Oracle,Oracle真的好方便!好了,不废话,这次准备记录一些 ...
- mysql各种集群的优缺点
mysql各种集群的优缺点 1.主从架构:只是有数据备份的功能: 2.主主互备+keepalived:实现数据备份加高可用: 3.主主互备,主主下面分别挂个从: 4.A和B主主互备,把从库都挂到B下, ...
- 《综合》MMM集群
<综合>MMM集群 部署集群基础环境 MySQL-MMM架构部署 MySQL-MMM架构使用 1 部署集群基础环境 1.1 问题 本案例要求为MySQL集群准备基础环境,完成以下任务操作: ...
- Shell脚本实现---Swarm集群部署实例（Swarm Cluster）
Shell脚本实现---Swarm集群部署实例(Swarm Cluster) 一.机器环境(均是centos7.8) IP hostname 角色 192.168.10.200 manager-swa ...
- MySQL数据库集群进行正确配置步骤
MySQL数据库集群进行正确配置步骤 2010-06-09 10:47 arrowcat 博客园 字号:T | T 我们今天是要和大家一起分享的是对MySQL数据库集群进行正确配置,我前两天在相关网站 ...
- Facebook揭密：如何让MySQL数据库集群自主运行
Facebook运行着全球最大的MySQL数据库集群,该集群分布在两个大洲上的多个数据中心中数以千计的服务器上.让人不解的是,Facebook只动用了一个很小的团队来管理这个庞大的MySQL数据库集群 ...
随机推荐
- mvn deploy 推送到私有仓库，注意当前日期
由于更改了本机系统时间到过去的一个时间,导致mvn deploy推送到私有仓库后,该更新的jar包时间戳比较旧,客户端不能更新得到新的jar包.
- RocketMQ 2主2从 集群搭建
安装环境 jdk1.7 alibaba-rocketmq-3.2.6.tar.gz VM虚拟机redhat6.5-x64:192.168.1.201 192.168.1.202 192.168.1. ...
- SQL语句常见优化方法
Sql优化方法 先进行选择运算(where limit)再进行连接运算 where子句中应把过滤性最强的条件放在最前面 where子句中字段的顺序应和组合索引中字段顺序一致 使用索引 使用覆盖索引来避 ...
- MySQL Crash Course #03# Chapter 5. 6 排序. BETWEEN. IS NULL
索引 排序检索的数据 SQL 过滤 vs. 应用程序过滤 简单 Where 补充:大小写敏感. BETWEEN. IS NULL Sorting Retrieved Data mysql> SE ...
- php检查是否是数字和字母
/* 检查是否是数字和字母* php内置函数ctype_alnum检查字符串是否是数字和字母,或者两者混合* $string*/ public function is_numandlitter($st ...
- Mysql错误:ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
昨晚添加完索引之后, 查询整表的时候抛出Lock wait timeout exceeded; try restarting transaction, 吓死小白的我, 为什么条件查询可以, 整表查不了 ...
- 不明原因报错集中处理:Undefined
1, NSGenericException错误 Terminating app due to uncaught exception 'NSGenericException', reason: '*** ...
- Centos7.2 安装Elasticsearch 6
下载 elasticsearch.6.0.0.tar.gz 迁移文件到usr/local中 mv elasticsearch-.tar.gz /usr/local/ cd /usr/local tar ...
- JavaScript:正则表达式 全局
关于正则表达式的 RegExp方法:test,exec, String 方法:match,search, 全局 g var str = "abababa"; var re = /a ...
- 史丰收速算|2014年蓝桥杯B组题解析第四题-fishers
史丰收速算 史丰收速算法的革命性贡献是:从高位算起,预测进位.不需要九九表,彻底颠覆了传统手算! 速算的核心基础是:1位数乘以多位数的乘法. 其中,乘以7是最复杂的,就以它为例. 因为,1/7 是个循 ...