Kubernetes v1.22 编译 kubeadm 修改证书有效期到 100 年

此方法支持以下 kubeadm版本

v1.22到v1.25

kubeadm 默认证书为一年，一年过期后，会导致 api service 不可用，使用过程中会出现：x509: certificate has expired or is not yet valid.

001、获取源码

访问：https://github.com/kubernetes/kubernetes/releases，下载特定版本源码

wget -c https://github.com/kubernetes/kubernetes/archive/v1.22.12.tar.gz

tar xf v1.22.12.tar.gz

mv kubernetes-1.22.12 kubernetes

cd kubernetes

002、修改证书有效期

主要有两个地方需要修改

021、修改 CA 有效期为 100 年（默认为 10 年）

vim ./staging/src/k8s.io/client-go/util/cert/cert.go

// 这个方法里面 NotAfter:              now.Add(duration365d * 10).UTC()

// 默认有效期就是 10 年，改成 100 年 (sysin)

// 输入 /NotAfter 查找，回车定位

func NewSelfSignedCACert(cfg Config, key crypto.Signer) (*x509.Certificate, error) {

        now := time.Now()

        tmpl := x509.Certificate{

                SerialNumber: new(big.Int).SetInt64(0),

                Subject: pkix.Name{

                        CommonName:   cfg.CommonName,

                        Organization: cfg.Organization,

                },

                NotBefore:             now.UTC(),

                // NotAfter:              now.Add(duration365d * 10).UTC(),

                NotAfter:              now.Add(duration365d * 100).UTC(),

                KeyUsage:              x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,

                BasicConstraintsValid: true,

                IsCA:                  true,

        }

        certDERBytes, err := x509.CreateCertificate(cryptorand.Reader, &tmpl, &tmpl, key.Public(), key)

        if err != nil {

                return nil, err

        }

        return x509.ParseCertificate(certDERBytes)

}

022、修改证书有效期为 100 年（默认为 1 年）

vim ./cmd/kubeadm/app/constants/constants.go

// 就是这个常量定义 CertificateValidity，改成 * 100 年 (sysin)

// 输入 /CertificateValidity 查找，回车定位

const (

        // KubernetesDir is the directory Kubernetes owns for storing various configuration files

        KubernetesDir = "/etc/kubernetes"

        // ManifestsSubDirName defines directory name to store manifests

        ManifestsSubDirName = "manifests"

        // TempDirForKubeadm defines temporary directory for kubeadm

        // should be joined with KubernetesDir.

        TempDirForKubeadm = "tmp"

        // CertificateValidity defines the validity for all the signed certificates generated by kubeadm

        // CertificateValidity = time.Hour * 24 * 365

        CertificateValidity = time.Hour * 24 * 365 * 100

        // CACertAndKeyBaseName defines certificate authority base name

        CACertAndKeyBaseName = "ca"

        // CACertName defines certificate name

        CACertName = "ca.crt"

        // CAKeyName defines certificate name

        CAKeyName = "ca.key"

验证一下已经正确修改：

cat ./staging/src/k8s.io/client-go/util/cert/cert.go | grep NotAfter

cat ./cmd/kubeadm/app/constants/constants.go | grep CertificateValidity

git 验证（可选，适用于 git 获取的源码），修改的内容如下：

git diff

diff --git a/cmd/kubeadm/app/constants/constants.go b/cmd/kubeadm/app/constants/constants.go

index 75adf43..54f25fa 100644

--- a/cmd/kubeadm/app/constants/constants.go

+++ b/cmd/kubeadm/app/constants/constants.go

@@ -44,7 +44,7 @@ const (

        TempDirForKubeadm = "tmp"

        // CertificateValidity defines the validity for all the signed certificates generated by kubeadm

-       CertificateValidity = time.Hour * 24 * 365

+       CertificateValidity = time.Hour * 24 * 365 * 100

        // CACertAndKeyBaseName defines certificate authority base name

        CACertAndKeyBaseName = "ca"

diff --git a/staging/src/k8s.io/client-go/util/cert/cert.go b/staging/src/k8s.io/client-go/util/cert/cert.go

index 9fd097a..865d6bb 100644

--- a/staging/src/k8s.io/client-go/util/cert/cert.go

+++ b/staging/src/k8s.io/client-go/util/cert/cert.go

@@ -63,7 +63,7 @@ func NewSelfSignedCACert(cfg Config, key crypto.Signer) (*x509.Certificate, erro

                        Organization: cfg.Organization,

                },

                NotBefore:             now.UTC(),

-               NotAfter:              now.Add(duration365d * 10).UTC(),

+               NotAfter:              now.Add(duration365d * 100).UTC(),

                KeyUsage:              x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,

                BasicConstraintsValid: true,

                IsCA:                  true,

源代码改好了，接下来就是编译 kubeadm 了。

003、编译

编译方式采用本机编译

环境需求参看下面的官方文档。

https://github.com/kubernetes/community/blob/master/contributors/devel/development.md

本例编译参考信息

cat /etc/redhat-release

CentOS Linux release 7.6.1810 (Core) 

uname -r

4.19.188-10.el7.ucloud.x86_64

4C 8G

031、软件包准备

yum groupinstall "Development Tools" -y #gcc, make etc.

yum install rsync jq -y

032、GoLang 环境

查看 kube-cross 的 TAG 版本号

cat ./build/build-image/cross/VERSION

v1.22.0-go1.16.15-buster.0

安装 Go 环境

# 这里下载go版本需要以 kube-cross 的 TAG 版本号 为准
wget -c https://golang.google.cn/dl/go1.16.15.linux-amd64.tar.gz

tar zxvf  go1.16.15.linux-amd64.tar.gz  -C /usr/local

# 编辑 / etc/profile 文件添加如下：

#go setting

export GOROOT=/usr/local/go

export GOPATH=/usr/local/gopath

export PATH=$PATH:$GOROOT/bin

#生效

source /etc/profile

验证

go version

go version go1.16.15 linux/amd64

033、编译

# 编译 kubeadm, 这里主要编译 kubeadm 即可

make all WHAT=cmd/kubeadm GOFLAGS=-v

# 编译 kubelet

# make all WHAT=cmd/kubelet GOFLAGS=-v

# 编译 kubectl

# make all WHAT=cmd/kubectl GOFLAGS=-v

#编译完产物在 _output/bin/kubeadm 目录下，

#其中 bin 是使用了软连接

#真实路径是_output/local/bin/linux/amd64/kubeadm

mv /usr/bin/kubeadm /usr/bin/kubeadm_backup

cp _output/local/bin/linux/amd64/kubeadm /usr/bin/kubeadm

chmod +x /usr/bin/kubeadm

查看编译后的信息

kubeadm version

kubeadm version: &version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.12", GitCommit:"b058e1760c79f46a834ba59bd7a3486ecf28237d", GitTreeState:"archive", BuildDate:"2022-12-07T02:57:53Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}

在其他节点上替换原有版本

mv /usr/bin/kubeadm /usr/bin/kubeadm_bak

# 把编译后的kubeadm 拷贝到 /usr/bin/目录下

004、更新证书

如果是使用原版 kubeadm 安装之后，可以手动执行命令更新证书有效期到 100 年。

可以先备份证书，证书在 /etc/kubernetes/pki

检查证书到期时间

kubeadm certs check-expiration


[check-expiration] Reading configuration from the cluster...

[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

W1207 16:16:32.988622  198420 utils.go:69] The recommended value for "clusterCIDR" in "KubeProxyConfiguration" is: 10.101.18.0/24; the provided value is: 10.101.16.0/23

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED

admin.conf                 Nov 13, 2122 06:19 UTC   99y             ca                      no

apiserver                  Nov 13, 2122 06:19 UTC   99y             ca                      no

apiserver-etcd-client      Nov 13, 2122 06:19 UTC   99y             etcd-ca                 no

apiserver-kubelet-client   Nov 13, 2122 06:19 UTC   99y             ca                      no

controller-manager.conf    Nov 13, 2122 06:19 UTC   99y             ca                      no

etcd-healthcheck-client    Nov 13, 2122 06:19 UTC   99y             etcd-ca                 no

etcd-peer                  Nov 13, 2122 06:19 UTC   99y             etcd-ca                 no

etcd-server                Nov 13, 2122 06:19 UTC   99y             etcd-ca                 no

front-proxy-client         Nov 13, 2122 06:19 UTC   99y             front-proxy-ca          no

scheduler.conf             Nov 13, 2122 06:19 UTC   99y             ca                      no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED

ca                      Nov 13, 2122 06:19 UTC   99y             no

etcd-ca                 Nov 13, 2122 06:19 UTC   99y             no

front-proxy-ca          Nov 13, 2122 06:19 UTC   99y             no

续订证书

kubeadm certs renew all

再次查看证书有效期，全部都 100 年了

kubeadm certs check-expiration

005、疑问

A：node节点的kubeadm需不需要更新

为了防止留坑，master和node都更新掉

B：

C：什么时候更新证书

最好是在执行 Kubeadm join之前重新编译kubeadm，然后进行替换。替换后不用重启可以直接执行 kubeadm join 命令

参考文档

https://sysin.cn/blog/kubernetes-kubeadm-cert-100y/#3-2-2-GoLang-%E7%8E%AF%E5%A2%83