ETCD集群+ TLS认证
目录
1、下载etcd cfssl相关命令
# 下载etcd cfssl相关命令
[root@1a32vla0168zzzz cfssl]# ll /app/etcd/bin/
cfssl cfssl-certinfo cfssljson etcd etcdctl etcdutl
[root@1a32vla0168zzzz appdeploy]# cd /app/etcd/cfssl/
[root@1a32vla0168zzzz cfssl]# ls
ca.csr ca-csr.json ca-key.pem ca.pem config.json server.csr server.json server-key.pem server.pem
[root@1a32vla0168zzzz cfssl]# cat config.json
{
"signing": {
"default": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "87600h"
}
}
}
[root@1a32vla0168zzzz cfssl]# cat ca-csr.json
{
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"O": "etcd",
"OU": "etcd",
"L": "apisix",
"ST": "apisix",
"C": "china"
}
],
"CN": "etcd"
}
[root@1a32vla0168zzzz cfssl]# cat server.json
{
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"O": "etcd",
"OU": "etcd",
"L": "apisix",
"ST": "apisix",
"C": "china"
}
],
"CN": "etcd",
"hosts": [
"xxx.xxx.xxx.5",
"xxx.xxx.xxx.6",
"xxx.xxx.xxx.7"
]
}
[root@1a32vla0168zzzz cfssl]# cfssl gencert --initca=true ca-csr.json | cfssljson --bare ca
[root@1a32vla0168zzzz cfssl]# cfssl gencert --ca ca.pem --ca-key ca-key.pem --config config.json server.json | cfssljson --bare server
# 复制 .pem文件至其他节点
2、配置文件 /app/etcd/conf/conf.yml
etcd01（etcd02和etcd03配置替换下ip即可），注意https http的配置细节
# vim /app/etcd/conf/conf.yml
name: etcd01
data-dir: /app/etcd/data
initial-advertise-peer-urls: https://xxx.xxx.xxx.5:2380
listen-peer-urls: https://xxx.xxx.xxx.5:2380
listen-client-urls: https://xxx.xxx.xxx.5:2379,http://127.0.0.1:2379
advertise-client-urls: https://xxx.xxx.xxx.5:2379
initial-cluster-token: apisix-etcd-cluster
initial-cluster: etcd01=https://xxx.xxx.xxx.5:2380,etcd02=https://xxx.xxx.xxx.6:2380,etcd03=https://xxx.xxx.xxx.7:2380
initial-cluster-state: new
# [security]
client-transport-security:
client-cert-auth: true
trusted-ca-file: /app/etcd/cfssl/ca.pem
cert-file: /app/etcd/cfssl/server.pem
key-file: /app/etcd/cfssl/server-key.pem
peer-transport-security:
client-cert-auth: true
trusted-ca-file: /app/etcd/cfssl/ca.pem
cert-file: /app/etcd/cfssl/server.pem
key-file: /app/etcd/cfssl/server-key.pem
3、用系统命令启动，创建 /usr/lib/systemd/system/etcd.service
[unix_http_server]
file=//app/etcd/supervisor/upervisor.sock ; the path to the socket file
[supervisord]
logfile=/app/etcd/supervisor/supervisor.log ; main log file; default $CWD/supervisord.log
logfile_maxbytes=50MB
logfile_backups=5
loglevel=info
pidfile=/app/etcd/supervisor/supervisord.pid ; supervisord pidfile; default supervisord.pid
nodaemon=false
silent=false
minfds=1024
minprocs=200
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///app/etcd/supervisor/supervisor.sock
[program:apisix-etcd]
directory = /app/etcd/
command=/app/etcd/bin/etcd --config-file=/app/etcd/conf/conf.yml
autostart=true
startsecs=5
startretries=3
autorestart=true ; 程序崩溃时自动重启，重启次数是有限制的，默认为3次
redirect_stderr=true ; 重定向输出的日志
stdout_syslog=true
stdout_logfile=/app/etcd/supervisor/stdout.log ; stdout log path, NONE for none; default AUTO
stdout_logfile_backups=5
stderr_logfile_backups=5
stderr_syslog=true
启动服务
systemctl daemon-reload && systemctl start etcd.service
注意：新集群中节点启动时，尽量快速同时启动所有节点，单独启动会报错
4、用 supervisor启动
[root@1a32vla0168zzzz etcd]# cat supervisor/supervisor.conf
[unix_http_server]
file=//app/etcd/supervisor/upervisor.sock ; the path to the socket file
[supervisord]
logfile=/app/etcd/supervisor/supervisor.log ; main log file; default $CWD/supervisord.log
logfile_maxbytes=50MB
logfile_backups=5
loglevel=info
pidfile=/app/etcd/supervisor/supervisord.pid ; supervisord pidfile; default supervisord.pid
nodaemon=false
silent=false
minfds=1024
minprocs=200
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///app/etcd/supervisor/supervisor.sock
[program:apisix-etcd]
directory = /app/etcd/
command=/app/etcd/bin/etcd --config-file=/app/etcd/conf/conf.yml
autostart=true
startsecs=5
startretries=3
autorestart=true ; 程序崩溃时自动重启，重启次数是有限制的，默认为3次
redirect_stderr=true ; 重定向输出的日志
stdout_syslog=true
stdout_logfile=/app/etcd/supervisor/stdout.log ; stdout log path, NONE for none; default AUTO
stdout_logfile_backups=5
stderr_logfile_backups=5
stderr_syslog=true
5、TLS验证
注意：https而不是http
[root@1a32vla0168zzzz etcd]# etcdctl endpoint health --endpoints "https://xxx.xxx.xxx.5:2379,https://xxx.xxx.xxx.6:2379,https://xxx.xxx.xxx.7:2379"
{"level":"warn","ts":1659374088.913921,"logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000032380/xxx.xxx.xxx.5:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: authentication handshake failed: x509: certificate signed by unknown authority\""}
{"level":"warn","ts":1659374088.9140902,"logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00032e700/xxx.xxx.xxx.6:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: authentication handshake failed: x509: certificate signed by unknown authority\""}
{"level":"warn","ts":1659374088.9141088,"logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000c08c0/xxx.xxx.xxx.7:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: authentication handshake failed: x509: certificate signed by unknown authority\""}
https://xxx.xxx.xxx.5:2379 is unhealthy: failed to commit proposal: context deadline exceeded
https://xxx.xxx.xxx.6:2379 is unhealthy: failed to commit proposal: context deadline exceeded
https://xxx.xxx.xxx.7:2379 is unhealthy: failed to commit proposal: context deadline exceeded
Error: unhealthy cluster
# 不带证书验证会发现提示都是unhealthy，必须带上证书
[root@1a32vla0168zzzz etcd]# etcdctl --cacert /app/etcd/cfssl/ca.pem --cert /app/etcd/cfssl/server.pem --key /app/etcd/cfssl/server-key.pem --endpoints "https://xxx.xxx.xxx.5:2379,https://xxx.xxx.xxx.6:2379,https://xxx.xxx.xxx.7:2379" endpoint health
https://xxx.xxx.xxx.5:2379 is healthy: successfully committed proposal: took = 9.169585ms
https://xxx.xxx.xxx.6:2379 is healthy: successfully committed proposal: took = 10.263672ms
https://xxx.xxx.xxx.7:2379 is healthy: successfully committed proposal: took = 13.415932ms
[root@1a32vla0168zzzz etcd]#
6、用户认证
[root@1a32vla0169zzzz etcd]# etcdctl auth status
Authentication Status: false
AuthRevision: 1
[root@1a32vla0169zzzz etcd]# etcdctl user list
[root@1a32vla0169zzzz etcd]# etcdctl role list
[root@1a32vla0169zzzz etcd]# etcdctl role add root
Role root created
[root@1a32vla0169zzzz etcd]# etcdctl user add root:123456
User root created
[root@1a32vla0169zzzz etcd]# etcdctl user grant-role root root
Role root is granted to user root
# 创建读写权限和仅读权限的用户和角色
[root@1a32vla0169zzzz etcd]# etcdctl role add read_write
Role read_write created
[root@1a32vla0169zzzz etcd]# etcdctl role add read_only
Role read_only created
[root@1a32vla0169zzzz etcd]# etcdctl role grant-permission read_write --prefix=true readwrite /
Role read_write updated
[root@1a32vla0169zzzz etcd]# etcdctl role grant-permission read_only --prefix=true read /
Role read_only updated
[root@1a32vla0169zzzz etcd]# etcdctl user add admin:admin123
User admin created
[root@1a32vla0169zzzz etcd]# etcdctl user add monitor:monitor123
User monitor created
[root@1a32vla0169zzzz etcd]# etcdctl user grant-role admin read_write
Role read_write is granted to user admin
[root@1a32vla0169zzzz etcd]# etcdctl user grant-role monitor read_only
Role read_only is granted to user monitor
# 开启用户认证
[root@1a32vla0169zzzz etcd]# etcdctl auth enable
Authentication Enabled
此时，验证需要带上用户密码
# 不带用户密码会报错
# etcdctl --cacert /app/etcd/cfssl/ca.pem --cert /app/etcd/cfssl/server.pem --key /app/etcd/cfssl/server-key.pem --endpoints "https://xxx.xxx.xxx.5:2379,https://xxx.xxx.xxx.6:2379,https://xxx.xxx.xxx.7:2379" endpoint health
{"level":"warn","ts":1659374402.6467996,"logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00040ca80/xxx.xxx.xxx.5:2379","attempt":0,"error":"rpc error: code = PermissionDenied desc = etcdserver: permission denied"}
{"level":"warn","ts":1659374402.6468987,"logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000b28c0/xxx.xxx.xxx.7:2379","attempt":0,"error":"rpc error: code = PermissionDenied desc = etcdserver: permission denied"}
{"level":"warn","ts":1659374402.6474524,"logger":"client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000321c0/xxx.xxx.xxx.6:2379","attempt":0,"error":"rpc error: code = PermissionDenied desc = etcdserver: permission denied"}
https://xxx.xxx.xxx.7:2379 is healthy: successfully committed proposal: took = 15.426246ms
https://xxx.xxx.xxx.5:2379 is healthy: successfully committed proposal: took = 15.216552ms
https://xxx.xxx.xxx.6:2379 is healthy: successfully committed proposal: took = 15.952318ms
# etcdctl --cacert /app/etcd/cfssl/ca.pem --cert /app/etcd/cfssl/server.pem --key /app/etcd/cfssl/server-key.pem --endpoints "https://xxx.xxx.xxx.5:2379,https://xxx.xxx.xxx.6:2379,https://xxx.xxx.xxx.7:2379" endpoint health --user=root:123456
https://xxx.xxx.xxx.6:2379 is healthy: successfully committed proposal: took = 933.418µs
https://xxx.xxx.xxx.5:2379 is healthy: successfully committed proposal: took = 915.707µs
https://xxx.xxx.xxx.7:2379 is healthy: successfully committed proposal: took = 1.209063ms
重点：新集群中节点启动时，尽量快速同时启动所有节点，单独启动会报错！！
ETCD集群+ TLS认证的更多相关文章
- 二进制搭建kubernetes多master集群【一、使用TLS证书搭建etcd集群】
上一篇我们介绍了kubernetes集群架构以及系统参数配置,参考:二进制搭建kubernetes多master集群[开篇.集群环境和功能介绍] 下面本文etcd集群才用三台centos7.5搭建完成 ...
- Kubernetes(k8s)集群部署（k8s企业级Docker容器集群管理）系列之自签TLS证书及Etcd集群部署（二）
0.前言 整体架构目录:ASP.NET Core分布式项目实战-目录 k8s架构目录:Kubernetes(k8s)集群部署(k8s企业级Docker容器集群管理)系列目录 一.服务器设置 1.把每一 ...
- Kubernetes-3.3：ETCD集群搭建及使用(https认证+数据备份恢复)
etcd集群搭建 环境介绍 基于CentOS Linux release 7.9.2009 (Core) ip hostname role 172.17.0.4 cd782d0a790b etcd1 ...
- 03-创建高可用 etcd 集群
本文档记录自己的学习历程! 创建高可用 etcd 集群 kuberntes 系统使用 etcd 存储所有数据,本文档介绍部署一个三节点高可用 etcd 集群的步骤,这三个节点使用以下机器: 192.1 ...
- Kubernetes1.91（K8s）安装部署过程（三）--创建高可用etcd集群
这里的etcd集群复用我们测试的3个节点,3个node都要安装并启动,注意修改配置文件 1.TLS认证文件分发:etcd集群认证用,除了本机有,分发到其他node节点 scp ca.pem kuber ...
- CentOS 7 ETCD集群配置大全
目录 前言 环境准备 安装 静态集群 配置 node01 配置文件 node02 配置文件 node03 配置文件 启动测试 查看集群状态 生成TLS证书 etcd证书创建 安装cfssl工具集 生成 ...
- k8s集群搭建之二：etcd集群的搭建
一 介绍 Etcd是一个高可用的 Key/Value 存储系统,主要用于分享配置和服务发现. 简单:支持 curl 方式的用户 API (HTTP+JSON) 安全:可选 SSL 客户端证书认证 快速 ...
- 分布式kv存储系统之Etcd集群
etcd是什么? etcd是一个高可用的分布式键值数据库,可用于服务发现,etcd采用 raft 一致性算法,基于 Go 语言实现.其特点有简单易用,所谓简单易用是指安装配置简单,提供http/htt ...
- K8s二进制部署单节点 etcd集群，flannel网络配置 ——锥刺股
K8s 二进制部署单节点 master --锥刺股 k8s集群搭建: etcd集群 flannel网络插件 搭建master组件 搭建node组件 1.部署etcd集群 2.Flannel 网络 ...
- etcd集群部署
etcd是用于共享配置和服务发现的分布式KV存储系统,随着CoreOS和Kubernetes等项目在开源社区日益火热,它们都用到了etcd组件作为一个高可用.强一致性的服务发现存储仓库.操作系统版本: ...
随机推荐
- [golang]查询ssl证书剩余有效天数并邮件提醒
前言 自从云厂商的免费ssl证书改成3个月,而且证书数量还是20个之后,自己网站的ssl证书就换成了其它免费方案.但是免费方案不会提醒证书过期,所以写个工具每天定时查询证书剩余有效天数,如果证书即将过 ...
- 用Python实现阿拉伯数字转换成中国汉字
要将阿拉伯数字转换成中国汉字表示的数字,我们需要一个映射表来转换每个数字,并且处理不同位数的数字(如十.百.千.万等). 1. Python实现阿拉伯数字转换成中国汉字 下面是一个完整的Python代 ...
- 全网最适合入门的面向对象编程教程：46 Python函数方法与接口-函数与事件驱动框架
全网最适合入门的面向对象编程教程:46 Python 函数方法与接口-函数与事件驱动框架 摘要: 函数是 Python 中的一等公民,是一种可重用的代码块,用于封装特定的逻辑:事件驱动框架是一种编程模 ...
- RxJS 系列 – 概念篇
前言 很长一段时间没有写 Angular 了 (哎...全栈的命),近期计划又要开始回去写了,于是就开始做复习咯. 我的复习是从 JS > TS > RxJS > Angular,与 ...
- Email 关于 POP3 IMAP SMTP office365 Outlook Gmail G-Suit shared mailbox小小理解
Outlook 是微软的一个 email 软件, 管理 email 的 UI. Gmail 是 google 的 office365 是一个配套, 里面有 email, one drive, exce ...
- Android Qcom USB Driver学习(三)
usb hub区分端口 USB 子系统拓扑浅析 USB ID Database VendorID and ProductID usb usb1: New USB device found, idVen ...
- USB协议详解第6讲（USB描述符-端点描述符）
1.USB描述符 USB描述符有设备描述符.标准配置描述符.接口描述符.端点描述符.字符串描述符,HID设备有HID描述符.报告描述符和物理描述符.今天主要是学习USB端点描述符的组成. 2.端点描述 ...
- 使用GrabCut做分割
主要完成了界面化设计,代码如下 import cv2 as cv import numpy as np import sys from PyQt5.Qt import * class MyWedige ...
- 最受DBA欢迎的数据库技术文档-巡检篇
有人说,"数据库巡检是数据库运维领域最重要的工作".的确,为了保证数据库的稳定.安全运行,除了可以对数据库进行监控以及时知晓故障苗头,定期的"健康体检"则尤为重 ...
- vagrant快速安装虚拟机