【云原生 · Kubernetes】kubernetes v1.23.3 二进制部署（三）

5 部署 etcd 集群

etcd 是基于 Raft 的分布式 KV 存储系统，由 CoreOS 开发，常用于服务发现、共享配置以及并发控制（如 leader 选举、分布式锁等）。
kubernetes 使用 etcd 集群持久化存储所有 API 对象、运行数据。

etcd 集群节点名称和 IP 如下：

集群节点名称	IP
k8s-master-1	192.168.2.175
k8s-master-2	192.168.2.176
k8s-master-3	192.168.2.178

注意：

1. 如果没有特殊指明，本文档的所有操作均在qist 节点上执行；

5.1 下载和分发 etcd 二进制文件

cd /opt/k8s/work
wget https://github.com/etcd-io/etcd/releases/download/v3.5.2/etcd-v3.5.2-linuxamd64.tar.gz
tar -xvf tcd-v3.5.2-linux-amd64.tar.gz

分发二进制文件到集群所有节点：

cd /opt/k8s/work
scp -r etcd-v3.5.2-linux-amd64/etcd* root@192.168.2.175:/apps/etcd/bin
scp -r etcd-v3.5.2-linux-amd64/etcd* root@192.168.2.176:/apps/etcd/bin
scp -r etcd-v3.5.2-linux-amd64/etcd* root@192.168.2.177:/apps/etcd/bin

5.2 创建 etcd 证书和私钥

• 创建etcd服务证书
• 创建证书签名请求：

cat > /opt/k8s/cfssl/etcd/etcd-server.json << EOF
{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "192.168.2.175","192.168.2.176","192.168.2.177",
    "k8s-master-1","k8s-master-2","k8s-master-3"
],
"key": {
  "algo": "rsa",
  "size": 2048
},
"names": [
  {
        "C": "CN",
        "ST": "GuangDong",
		"L": "GuangZhou",
		"O": "k8s",
		"OU": "Qist"
    }
  ]
}
EOF

生成证书和私钥：

cfssl gencert \
    -ca=/opt/k8s/cfssl/pki/etcd/etcd-ca.pem \
    -ca-key=/opt/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
    -config=/opt/k8s/cfssl/ca-config.json \
    -profile=kubernetes \
    /opt/k8s/cfssl/etcd/etcd-server.json | \
    cfssljson -bare /opt/k8s/cfssl/pki/etcd/etcd-server

• 创建etcd节点证书

192.168.2.175节点

cat > /opt/k8s/cfssl/etcd/k8s-master-1.json << EOF
{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "192.168.2.175",
    "k8s-master-1"
],
"key": {
    "algo": "rsa",
    "size": 2048
},
"names": [
  {
        "C": "CN",
		"ST": "GuangDong",
		"L": "GuangZhou",
		"O": "k8s",
		"OU": "Qist"
    }
  ]
}
EOF

生成证书和私钥：

cfssl gencert \
    -ca=/opt/k8s/cfssl/pki/etcd/etcd-ca.pem \
	-ca-key=/opt/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
	-config=/opt/k8s/cfssl/ca-config.json \
	-profile=kubernetes \
	/opt/k8s/cfssl/etcd/k8s-master-1.json | \
	cfssljson -bare /opt/k8s/cfssl/pki/etcd/etcd-member-k8s-master-1

192.168.2.176节点

cat > /opt/k8s/cfssl/etcd/k8s-master-2.json << EOF
{
  "CN": "etcd",
  "hosts": [
	"127.0.0.1",
	"192.168.2.176",
	"k8s-master-2"
],
192.168.2.176节点
"key": {
  "algo": "rsa",
  "size": 2048
},
"names": [
  {
		"C": "CN",
		"ST": "GuangDong",
		"L": "GuangZhou",
		"O": "k8s",
		"OU": "Qist"
	}
  ]
}
EOF

生成证书和私钥：

cfssl gencert \
	-ca=/opt/k8s/cfssl/pki/etcd/etcd-ca.pem \
	-ca-key=/opt/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
	-config=/opt/k8s/cfssl/ca-config.json \
	-profile=kubernetes \
	/opt/k8s/cfssl/etcd/k8s-master-2.json | \
	cfssljson -bare /opt/k8s/cfssl/pki/etcd/etcd-member-k8s-master-2

192.168.2.177 节点

cat > /opt/k8s/cfssl/etcd/k8s-master-3.json << EOF
{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "192.168.2.177",
"k8s-master-3"
],
"key": {
	"algo": "rsa",
	"size": 2048
},
"names": [
	{
		"C": "CN",
		"ST": "GuangDong",
		"L": "GuangZhou",
		"O": "k8s",
		"OU": "Qist"
	}
  ]
}
EOF

生成证书和私钥：

cfssl gencert \
		-ca=/opt/k8s/cfssl/pki/etcd/etcd-ca.pem \
		-ca-key=/opt/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
		-config=/opt/k8s/cfssl/ca-config.json \
		-profile=kubernetes \
		/opt/k8s/cfssl/etcd/etcd-client.json | \
		cfssljson -bare /opt/k8s/cfssl/pki/etcd/etcd-client

分发生成的证书和私钥到各 etcd 节点：

# 分发server 证书
scp -r /opt/k8s/cfssl/pki/etcd/etcd-server* root@192.168.2.175:/apps/etcd/ssl
scp -r /opt/k8s/cfssl/pki/etcd/etcd-server* root@192.168.2.176:/apps/etcd/ssl
scp -r /opt/k8s/cfssl/pki/etcd/etcd-server* root@192.168.2.177:/apps/etcd/ssl
# 分发192.168.2.175 节点证书
scp -r /opt/k8s/cfssl/pki/etcd/etcd-member-k8s-master-1*
root@192.168.2.175:/apps/etcd/ssl
# 分发192.168.2.176 节点证书
scp -r /opt/k8s/cfssl/pki/etcd/etcd-member-k8s-master-2*
root@192.168.2.176:/apps/etcd/ssl
# 分发192.168.2.177 节点证书
scp -r /opt/k8s/cfssl/pki/etcd/etcd-member-k8s-master-3*
root@192.168.2.175:/apps/etcd/ssl
# 分发客户端证书到K8S master 节点 kube-apiserver 连接etcd 集群使用
scp -r /opt/k8s/cfssl/pki/etcd/etcd-client* root@192.168.2.175:/apps/k8s/ssl/etcd/
scp -r /opt/k8s/cfssl/pki/etcd/etcd-client* root@192.168.2.176:/apps/k8s/ssl/etcd/
scp -r /opt/k8s/cfssl/pki/etcd/etcd-client* root@192.168.2.177:/apps/k8s/ssl/etcd/

• WorkingDirectory 、 --data-dir ：指定工作目录和数据目录为${ETCD_DATA_DIR}，需在启动服务前创建这个目录；
• –wal-dir ：指定 wal 目录，为了提高性能，一般使用 SSD 或者和 --data-dir 不同的磁盘；
• –name ：指定节点名称，当 --initial-cluster-state 值为 new 时， --name 的参数值必须位于 --initial-cluster 列表中；
• –cert-file 、 --key-file ：etcd server 与 client 通信时使用的证书和私钥；
• –trusted-ca-file ：签名 client 证书的 CA 证书，用于验证 client 证书；
• –peer-cert-file 、 --peer-key-file ：etcd 与 peer 通信使用的证书和私钥；
• –peer-trusted-ca-file ：签名 peer 证书的 CA 证书，用于验证 peer 证书；

5.3 创建etcd 运行用户

k8s-master-1 k8s-master-2 k8s-master-3 节点上执行

• 创建etcd用户

useradd etcd -s /sbin/nologin -M

• etcd 目录给用户权限

chown -R etcd:etcd /apps/etcd
[root@k8s-master-3 ~]# ls -la /apps/etcd/
total 4
drwxr-xr-x 7 etcd etcd 64 Feb 10 20:32 .
drwxr-xr-x. 8 root root 85 Aug 26 18:54 ..
drwxr-xr-x 3 etcd etcd 117 Feb 10 20:28 bin
drwxr-xr-x 2 etcd etcd 18 Feb 10 20:33 conf
drwxr-xr-x 3 etcd etcd 26 Aug 26 12:57 data
drwxr-xr-x 2 etcd etcd 4096 Aug 26 12:58 ssl

5.4 启动 etcd 服务

k8s-master-1 k8s-master-2 k8s-master-3 节点上执行

# 全局刷新service
systemctl daemon-reload
# 设置etcd 开机启动
systemctl enable etcd
#重启etcd
systemctl restart etcd

• 必须先创建 etcd 数据目录和工作目录;
• etcd 进程首次启动时会等待其它节点的 etcd 加入集群，命令 systemctl start etcd 会卡住一段时 间，为正常现象；

5.5 检查启动结果

k8s-master-1 k8s-master-2 k8s-master-3 节点上执行

systemctl status etcd|grep Active
[root@k8s-master-1 conf]# systemctl status etcd|grep Active
Active: active (running) since Fri 2022-02-11 13:49:37 CST; 4h 5min ago
[root@k8s-master-2 ~]# systemctl status etcd|grep Active
Active: active (running) since Fri 2022-02-11 13:49:36 CST; 4h 4min ago
[root@k8s-master-3 ~]# systemctl status etcd|grep Active
Active: active (running) since Fri 2022-02-11 13:49:36 CST; 4h 5min ago

期待下次的分享，别忘了三连支持博主呀~
我是 念舒_C.ying ，期待你的关注~