【云原生 · Kubernetes】部署高可用kube-scheduler集群

个人名片：
因为云计算成为了监控工程师‍
个人博客：念舒_C.ying
CSDN主页️：念舒_C.ying

部署高可用kube-scheduler集群

该集群包含 3 个节点，启动后将通过竞争选举机制产生一个 leader 节点，其它节点为阻塞状态。当leader 节点不可用后，剩余节点将再次进行选举产生新的 leader 节点，从而保证服务的可用性。

先生成 x509 证书和私钥，kube-scheduler 在如下两种情况下使用该证书：

与 kube-apiserver 的安全端口通信;
在安全端口(https，10259) 输出 prometheus 格式的 metrics；

13.1 创建 kube-scheduler 证书和私钥

创建证书签名请求：

cd /opt/k8s/work

cat > /opt/k8s/cfssl/k8s/k8s-scheduler.json <<EOF

{

"CN": "system:kube-scheduler",

"hosts": [""],

"key": {

"algo": "rsa",

"size": 2048

},

"names": [

{

"C": "CN",

"ST": "GuangDong",

"L": "GuangZhou",

"O": "system:kube-scheduler",

"OU": "Kubernetes-manual"

}

]

}

EOF

hosts 列表包含所有 kube-scheduler 节点 IP；
CN 和 O 均为 system:kube-scheduler ，kubernetes 内置的 ClusterRoleBindings system:kubescheduler 将赋予kube-scheduler 工作所需的权限；

生成证书和私钥：

cd /opt/k8s/work

cfssl gencert \

-ca=/opt/k8s/cfssl/pki/k8s/k8s-ca.pem \

-ca-key=/opt/k8s/cfssl/pki/k8s/k8s-ca-key.pem \

-config=/opt/k8s/cfssl/ca-config.json \

-profile=kubernetes \

/opt/k8s/cfssl/k8s/k8s-scheduler.json | \

cfssljson -bare /opt/k8s/cfssl/pki/k8s/k8s-scheduler

ls /opt/k8s/cfssl/pki/k8s/k8s-scheduler*pem

将生成的证书和私钥分发到所有 master 节点：

cd /opt/k8s/work

scp -r /opt/k8s/cfssl/pki/k8s/k8s-scheduler* root@192.168.2.175:/apps/k8s/ssl/k8s

scp -r /opt/k8s/cfssl/pki/k8s/k8s-scheduler* root@192.168.2.176:/apps/k8s/ssl/k8s

scp -r /opt/k8s/cfssl/pki/k8s/k8s-scheduler* root@192.168.2.177:/apps/k8s/ssl/k8s

13.2 创建和分发 kubeconfig 文件

kube-scheduler 使用 kubeconfig 文件访问 apiserver，该文件提供了 apiserver 地址、嵌入的 CA 证书和
kube-scheduler 证书：

cd /opt/k8s/kubeconfig

# 设置集群参数

kubectl config set-cluster kubernetes \

--certificate-authority=/opt/k8s/cfssl/pki/k8s/k8s-ca.pem \

--embed-certs=true \

--server=https://127.0.0.1:6443 \

--kubeconfig=kube-scheduler.kubeconfig

# 设置客户端认证参数

kubectl config set-credentials system:kube-scheduler \

--client-certificate=/opt/k8s/cfssl/pki/k8s/k8s-scheduler.pem \

--embed-certs=true \

--client-key=/opt/k8s/cfssl/pki/k8s/k8s-scheduler-key.pem \

--kubeconfig=kube-scheduler.kubeconfig

# 设置上下文参数

kubectl config set-context kubernetes \

--cluster=kubernetes \

--user=system:kube-scheduler \

--kubeconfig=kube-scheduler.kubeconfig

# 设置默认上下文

kubectl config use-context kubernetes --kubeconfig=kube-scheduler.kubeconfig

分发 kubeconfig 到所有 master 节点：

cd /opt/k8s/kubeconfig

scp kube-scheduler.kubeconfig root@192.168.2.175:/apps/k8s/config/

scp kube-scheduler.kubeconfig root@192.168.2.176:/apps/k8s/config/

scp kube-scheduler.kubeconfig root@192.168.2.177:/apps/k8s/config/

13.3 创建 kube-scheduler 配置文件

cd /opt/k8s/work

cat >kube-scheduler <<EOF

KUBE_SCHEDULER_OPTS=" \

--logtostderr=true \

--bind-address=0.0.0.0 \

--leader-elect=true \

--kubeconfig=/apps/k8s/config/kube-scheduler.kubeconfig \

--authentication-kubeconfig=/apps/k8s/config/kubescheduler.kubeconfig \

--authorization-kubeconfig=/apps/k8s/config/kubescheduler.kubeconfig \

--tls-cert-file=/apps/k8s/ssl/k8s/k8s-scheduler.pem \

--tls-private-key-file=/apps/k8s/ssl/k8s/k8s-scheduler-key.pem \

--client-ca-file=/apps/k8s/ssl/k8s/k8s-ca.pem \

--requestheader-allowed-names= \

--requestheader-extra-headers-prefix=X-Remote-Extra- \

--requestheader-group-headers=X-Remote-Group \

--requestheader-username-headers=X-Remote-User \

--alsologtostderr=true \

--kube-api-qps=100 \

--authentication-tolerate-lookup-failure=false \

--kube-api-burst=100 \

--log-dir=/apps/k8s/log \

--tls-ciphersuites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,

TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDH

E_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES

_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256 \

--v=2"

EOF

–kubeconfig ：指定 kubeconfig 文件路径，kube-scheduler 使用它连接和验证 kube-apiserver；
–leader-elect=true ：集群运行模式，启用选举功能；被选为 leader 的节点负责处理工作，其它节点为阻塞状态；

分发 kube-scheduler 配置文件到所有 master 节点

cd /opt/k8s/work

scp kube-scheduler root@192.168.2.175:/apps/k8s/conf/

scp kube-scheduler root@192.168.2.176:/apps/k8s/conf/

scp kube-scheduler root@192.168.2.177:/apps/k8s/conf/

13.4 创建 kube-scheduler systemd unit 模板文件

cd /opt/k8s/work

cat > kube-scheduler.service <<EOF

[Unit]

Description=Kubernetes Scheduler

Documentation=https://github.com/kubernetes/kubernetes

[Service]

LimitNOFILE=655350

LimitNPROC=655350

LimitCORE=infinity

LimitMEMLOCK=infinity

EnvironmentFile=-/apps/k8s/conf/kube-scheduler

ExecStart=/apps/k8s/bin/kube-scheduler \$KUBE_SCHEDULER_OPTS

Restart=on-failure

RestartSec=5

[Install]

WantedBy=multi-user.target

EOF

13.5 为各节点创建和分发 kube-scheduler systemd unit 文件

分发到所有 master 节点：

cd /opt/k8s/work

scp kube-scheduler.service root@192.168.2.175:/usr/lib/systemd/system/

scp kube-scheduler.service root@192.168.2.176:/usr/lib/systemd/system/

scp kube-scheduler.service root@192.168.2.177:/usr/lib/systemd/system/

13.6 启动 kube-scheduler 服务

k8s-master-1 k8s-master-2 k8s-master-3 节点上执行

# 全局刷新service

systemctl daemon-reload

# 设置kube-scheduler开机启动

systemctl enable kube-scheduler

#重启kube-scheduler

systemctl restart kube-scheduler

13.7 检查服务运行状态

k8s-master-1 k8s-master-2 k8s-master-3 节点上执行

systemctl status kube-scheduler|grep Active

确保状态为 active (running) ，否则查看日志，确认原因：

journalctl -u kube-scheduler

kube-scheduler 监听 10259 端口，接收 https 请求：

[root@k8s-master-3 conf]# netstat -tnlp| grep kube-sc

tcp6 0 0 :::10259 :::* LISTEN

1887/kube-scheduler

13.8 查看当前的 leader

kubectl -n kube-system get leases kube-scheduler

root@Qist work# kubectl -n kube-system get leases kube-scheduler

NAME HOLDER AGE

kube-scheduler k8s-master-2_383bedd9-26ec-40c3-95e6-182aebe9b1b9 1d

13.9 测试 kube-scheduler 集群的高可用

随便找一个或两个 master 节点，停掉 kube-scheduler 服务，看其它节点是否获取了 leader 权限。

期待下次的分享，别忘了三连支持博主呀~
我是 念舒_C.ying ，期待你的关注~

附专栏链接
【云原生 · Kubernetes】runtime组件
 【云原生 · Kubernetes】apiserver高可用
 【云原生 · Kubernetes】kubernetes v1.23.3 二进制部署（三）
【云原生 · Kubernetes】kubernetes v1.23.3 二进制部署（二）
【云原生 · Kubernetes】kubernetes v1.23.3 二进制部署（一）
【云原生 · Kubernetes】Kubernetes 编排部署GPMall（一）