获取redis实例绑定cpu的情况

redis是一个单线模型的nosql类型的数据库，而目前接触到的服务器大都是多核的，比如8c，16c,32c，64c等等。为了充分利用主机，在一台主机上必然会部署多个redis实例，默认情况cpu会随机选择，但经过观察自动选择的时候很大情况下会选择同一逻辑cpu，这样导致cpu使用不均衡，撑得死了，饿的饿死了，怎么破。

其实可以对指定的进程id进行cpu绑定，绑定前的情况如下：

27001 instance bind on cpu  0-7
27002 instance bind on cpu  0-7
27003 instance bind on cpu  0-7
27004 instance bind on cpu  0-7
27005 instance bind on cpu  0-7
27007 instance bind on cpu  0-7
27008 instance bind on cpu  0-7

生成绑定的命令

ps aux|grep redis-server |grep -v grep |awk 'BEGIN {i=0}{i++}{print "taskset -pc " i, $2}'

　　

绑定后的情况如下：

27001 instance bind on cpu  1
27002 instance bind on cpu  2
27003 instance bind on cpu  3
27004 instance bind on cpu  4
27005 instance bind on cpu  5
27007 instance bind on cpu  6
27008 instance bind on cpu  7

绑定后获取cpu绑定情况的确认脚本如下：

ps -ef |grep redis-server |grep -v grep | awk '$NF~/cluster/{print $2,$(NF-1);next}{print $2,$NF}' > pid_instance

cat pid_instance | while read line
do
	pid=$(echo $line | awk '{print $1}')
	port=$(echo $line | awk -F':' '{print $2}')
	echo "$port instance bind on cpu `taskset -pc $pid | awk -F':' '{print $2}'`"
done
rm -rf pid_instance

如果有很多台redis实例，可以通过ansible分发该脚本到对应的主机上，然后跑一下sh redis_cpu.sh xxx脚本,xxx文件中是以all为分组的redis主机列表，例如：

cat >redis_cpu.sh <<EOF
#!/bin/bash

ansible -i $1 all -m copy -s -a "src=./get_redis_bind_cpu.sh dest=/tmp/get_redis_bind_cpu.sh"
ansible -i $1 all -m shell -s -a "sh get_redis_bind_cpu.sh"
EOF

cat >get_redis_bind_cpu.sh <<EOF
#!/bin/bash

ps -ef |grep redis-server |grep -v grep | awk '$NF~/cluster/{print $2,$(NF-1);next}{print $2,$NF}' > pid_instance

cat pid_instance | while read line
do
        pid=$(echo $line | awk '{print $1}')
        port=$(echo $line | awk -F':' '{print $2}')
        echo "$port instance bind on cpu `taskset -pc $pid | awk -F':' '{print $2}'`"
done
rm -rf pid_instance
EOF

　

添加一下输出优化，直观看出来是不是绑定了cpu

ps -ef |grep redis-server |grep -v grep | awk '$NF~/cluster/{print $2,$(NF-1);next}{print $2,$NF}' > pid_instance

cat pid_instance | while read line
do
        pid=$(echo $line | awk '{print $1}')
        port=$(echo $line | awk -F':' '{print $2}')
		bind_current=$(taskset -pc $pid | awk -F':' '{print $2}')
		total=$(cat /proc/cpuinfo |grep processor |wc -l)
		start=0
		let end=total-1
		bind_default="$start-$end"
		if [[ $bind_current -ne $bind_default ]];then
			echo "$port instance bind on cpu $bind_current ok"
		else
			echo "$port instance not set bind cpu default $bind_default,please check!!!"
		fi
done
rm -rf pid_instance

如果没有绑定：

[root@testdb1 ~]# sh c.sh
6379 instance not set bind cpu default 0-23,please check!!!
29009 instance not set bind cpu default 0-23,please check!!!
29095 instance not set bind cpu default 0-23,please check!!!
27000 instance not set bind cpu default 0-23,please check!!!
27001 instance not set bind cpu default 0-23,please check!!!
29001 instance not set bind cpu default 0-23,please check!!!
29002 instance not set bind cpu default 0-23,please check!!!
29003 instance not set bind cpu default 0-23,please check!!!
29004 instance not set bind cpu default 0-23,please check!!!
29005 instance not set bind cpu default 0-23,please check!!!
29006 instance not set bind cpu default 0-23,please check!!!
29007 instance not set bind cpu default 0-23,please check!!!
29008 instance not set bind cpu default 0-23,please check!!!
29000 instance not set bind cpu default 0-23,please check!!!

如果有绑定

27183 instance bind on cpu  1 ok
27184 instance bind on cpu  2 ok
27185 instance bind on cpu  3 ok
27186 instance bind on cpu  4 ok
27187 instance bind on cpu  5 ok
27188 instance bind on cpu  6 ok
27189 instance bind on cpu  7 ok
27190 instance bind on cpu  8 ok
27191 instance bind on cpu  9 ok
27192 instance bind on cpu  10 ok
27193 instance bind on cpu  11 ok
27194 instance bind on cpu  11 ok
27195 instance bind on cpu  10 ok
27196 instance bind on cpu  9 ok
27197 instance bind on cpu  8 ok

@20190509

cat >get_redis_bind_cpu.sh <<EOF
#!/bin/bash

ps -ef |grep redis-server |grep -v grep | awk '\$NF~/cluster/{print \$2,\$(NF-1);next}{print \$2,\$NF}' > pid_instance

cat pid_instance | while read line
do
        pid=\$(echo \$line | awk '{print \$1}')
        port=\$(echo \$line | awk -F':' '{print \$2}')
        bind_current=\$(taskset -pc \$pid | awk -F':' '{print \$2}')
        total=\$(cat /proc/cpuinfo |grep processor |wc -l)
        start=0
        let end=total-1
        bind_default="\$start-\$end"
        if [[ \$bind_current -ne \$bind_default ]];then
            echo "\$port instance bind on cpu \$bind_current ok"
        else
            echo "\$port instance not set bind cpu default \$bind_default,please check!!!" >> no_bind.log
        fi
done
rm -rf pid_instance
EOF

cat >redis_cpu.sh <<EOF
#!/bin/bash

ansible -i \$1 all -m copy -s -a "src=./get_redis_bind_cpu.sh dest=/tmp/get_redis_bind_cpu.sh" > /dev/null
ansible -i \$1 all -m shell -s -a "cd /tmp;sh get_redis_bind_cpu.sh" > /dev/null
ansible -i \$1 all -m shell -s -a "ls -l /tmp/no_bind.log 2>/dev/null"

EOF

PS:
1.执行前确保ansible到目标所有redis实例的机器可以跑通。
2.执行sh redis_cpu.sh redis_host  |grep -v FAILED
3.有输出的即是对应主机上redis实例存在没有绑定cpu的，具体实例端口在对应机器的/tmp/no_bind.log