kubespray续签k8s证书

查看证书过期时期

[root@node1 ~]# openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text |grep ' Not '
            Not Before: Sep  4 08:29:00 2019 GMT
            Not After : Sep  3 08:29:02 2020 GMT

备份etcd

[root@node1 etcd-backup]# export ETCDCTL_API=3
[root@node1 etcd-backup]# etcdctl snapshot save "/root/$(date +%Y%m%d_%H%M%S)_snapshot.db" --cacert=/etc/ssl/etcd/ssl/ca.pem --cert=/etc/ssl/etcd/ssl/node-node1.pem --key=/etc/ssl/etcd/ssl/node-node1-key.pem --endpoints=https://192.168.10.132:2379
Snapshot saved at /root/20200814_142210_snapshot.db

master1备份证书

cp -ar /etc/kubernetes{,.bak}

Kubespray部署的k8s会生成以下证书

K8s组件之间认证需要的证书

ca.crt ca.key

apiserver.crt apiserver.key

apiserver-kubelet-client.crt apiserver-kubelet-client.key

front-proxy-ca.crt front-proxy-ca.key front-proxy-client.crt front-proxy-client.key

sa.key sa.pub

Etcd认证的证书

ca.pem ca-key.pem

admin-node*.pem admin-node*-key.pem

member-node*.pem member-node*-key.pem

node-node*.pem node-node*-key.pem

ca.crt默认是10年，apiserver.crt apiserver-kubelet-client.crt默认一年，front-proxy-ca.crt是独立的ca证书，默认是10年， front-proxy-client.crt默认1年。Etcd的证书默认是100年

我们只需要更新apiserver.crt apiserver-kubelet-client.crt front-proxy-client即可

Master1节点重新生成证书，并同步证书至其他master节点

kubeadm alpha certs renew apiserver --config "/etc/kubernetes/kubeadm-config.yaml"
kubeadm alpha certs renew apiserver-kubelet-client --config "/etc/kubernetes/kubeadm-config.yaml"
kubeadm alpha certs renew front-proxy-client --config "/etc/kubernetes/kubeadm-config.yaml"

删除所有主机组件之间认证的kubeconfig

Kubeconfig是k8s其他组件与apiserver通信的认证密钥，apiserver证书更新了，这些密钥文件都需要更新。

ansible -i /tools/kubespray/inventory/inventory.cfg all -m shell -a "cd /etc/kubernetes && rm -rf admin.conf scheduler.conf controller-manager.conf kubelet.conf bootstrap-kubelet.conf"

Master节点分别再次生成kubeconfig（所有master节点都需要执行）

kubeadm init phase kubeconfig all --config "/etc/kubernetes/kubeadm-config.yaml"

安装有kubectl的节点覆盖config

\cp /etc/kubernetes/admin.conf /root/.kube/config

master1节点重启k8s相关组件

docker ps |grep apiserver|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

docker ps |grep kube-scheduler|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

docker ps |grep kube-controller|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

systemctl restart kubelet

拷贝master1的证书到master2上

scp -r /etc/kubernetes/ssl root@192.168.10.133:/etc/kubernetes/

master2上删除旧配置

cd /etc/kubernetes && rm -rf admin.conf scheduler.conf controller-manager.conf kubelet.conf bootstrap-kubelet.conf

重新生成新配置

kubeadm init phase kubeconfig all --config "/etc/kubernetes/kubeadm-config.yaml"

master2节点重启k8s相关组件

docker ps |grep apiserver|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

docker ps |grep kube-scheduler|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

docker ps |grep kube-controller|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

systemctl restart kubelet

查看节点状态

kubectl get node

worker node节点无需操作，证书会通过kubelet自动更新，各节点执行以下命令验证续签是否生效：

# Cert from api-server

echo -n | openssl s_client -connect localhost:6443 2>&1 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | openssl x509 -text -noout | grep Not

# Cert from controller manager

echo -n | openssl s_client -connect localhost:10257 2>&1 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | openssl x509 -text -noout | grep Not

# Cert from scheduler

echo -n | openssl s_client -connect localhost:10259 2>&1 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | openssl x509 -text -noout | grep Not