K8s集群部署（二）------ Master节点部署

Master节点要部署三个服务：API Server、Scheduler、Controller Manager。

apiserver提供集群管理的REST API接口，包括认证授权、数据校验以 及集群状态变更等

　　只有API Server才直接操作etcd

　　其他模块通过API Server查询或修改数据

　　提供其他模块之间的数据交互和通信的枢纽

scheduler负责分配调度Pod到集群内的node节点

　   监听kube-apiserver，查询还未分配Node的Pod

　　根据调度策略为这些Pod分配节点

controller-manager由一系列的控制器组成，它通过apiserver监控整个 集群的状态，并确保集群处于预期的工作状态

　　　　　

　

1.部署Kubernetes API服务部署

0.准备软件包

cd /usr/local/src/kubernetes
cp server/bin/kube-apiserver /opt/kubernetes/bin/  \
cp server/bin/kube-controller-manager /opt/kubernetes/bin/ \
cp server/bin/kube-scheduler /opt/kubernetes/bin/

1.创建生成CSR的 JSON 配置文件

[root@k8s-master kubernetes]# cd /usr/local/src/ssl/
[root@k8s-master ssl]# vim kubernetes-csr.json
{
  "CN": "kubernetes",
  "hosts": [
    "127.0.0.1",
    "10.0.3.225",     #Master IP地址
    "10.1.0.1",       ？？？
    "kubernetes",
    "kubernetes.default",
    "kubernetes.default.svc",
    "kubernetes.default.svc.cluster",
    "kubernetes.default.svc.cluster.local"
  ],
  "key": {
    "algo": "rsa",
    "size":
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

2.生成 kubernetes 证书和私钥

cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
   -ca-key=/opt/kubernetes/ssl/ca-key.pem \
   -config=/opt/kubernetes/ssl/ca-config.json \
   -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes

#拷贝证书到其他节点
cp kubernetes*.pem /opt/kubernetes/ssl/
scp kubernetes*.pem 10.0.3.226:/opt/kubernetes/ssl/
scp kubernetes*.pem 10.0.3.227:/opt/kubernetes/ssl/

3.创建 kube-apiserver 使用的客户端 token 文件

[root@k8s-master ssl]# head -c  /dev/urandom | od -An -t x | tr -d ' '
4c7d89749d1e1a15e5fe55eb5e8446ec
[root@k8s-master ssl]# vim /opt/kubernetes/ssl/bootstrap-token.csv
4c7d89749d1e1a15e5fe55eb5e8446ec,kubelet-bootstrap,,"system:kubelet-bootstrap"

4.创建基础用户名/密码认证配置

vim /opt/kubernetes/ssl/basic-auth.csv
admin,admin,
readonly,readonly,

5.部署Kubernetes API Server

vim /usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
ExecStart=/opt/kubernetes/bin/kube-apiserver \
  --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota,NodeRestriction \
  --bind-address=10.0.3.225 \
  --insecure-bind-address=127.0.0.1 \
  --authorization-mode=Node,RBAC \
  --runtime-config=rbac.authorization.k8s.io/v1 \
  --kubelet-https=true \
  --anonymous-auth=false \
  --basic-auth-file=/opt/kubernetes/ssl/basic-auth.csv \
  --enable-bootstrap-token-auth \
  --token-auth-file=/opt/kubernetes/ssl/bootstrap-token.csv \
  --service-cluster-ip-range=10.1.0.0/ \
  --service-node-port-range=- \
  --tls-cert-file=/opt/kubernetes/ssl/kubernetes.pem \
  --tls-private-key-file=/opt/kubernetes/ssl/kubernetes-key.pem \
  --client-ca-file=/opt/kubernetes/ssl/ca.pem \
  --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --etcd-cafile=/opt/kubernetes/ssl/ca.pem \
  --etcd-certfile=/opt/kubernetes/ssl/kubernetes.pem \
  --etcd-keyfile=/opt/kubernetes/ssl/kubernetes-key.pem \
  --etcd-servers=https://10.0.3.225:2379,https://10.0.3.226:2379,https://10.0.3.227:2379 \
  --enable-swagger-ui=true \
  --allow-privileged=true \
  --audit-log-maxage= \
  --audit-log-maxbackup= \
  --audit-log-maxsize= \
  --audit-log-path=/opt/kubernetes/log/api-audit.log \
  --event-ttl=1h \
  --v= \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log
Restart=on-failure
RestartSec=
Type=notify
LimitNOFILE=

[Install]
WantedBy=multi-user.target

6.启动API Server服务

systemctl daemon-reload
systemctl enable kube-apiserver
systemctl start kube-apiserver

查看API Server服务状态
systemctl status kube-apiserver

　[root@k8s-master ssl]# netstat -lntup|grep kube-apiser
  tcp 0 0 10.0.3.225:6443 0.0.0.0:* LISTEN 27784/kube-apiserve    
  tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 27784/kube-apiserve

部署ControllerManager服务

刚才安装包已经拷贝过去了，直接配置系统服务即可。

[root@k8s-master ssl]# vim /usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
ExecStart=/opt/kubernetes/bin/kube-controller-manager \
  --address=127.0.0.1 \
  --master=http://127.0.0.1:8080 \
  --allocate-node-cidrs=true \
  --service-cluster-ip-range=10.1.0.0/ \
  --cluster-cidr=10.2.0.0/ \
  --cluster-name=kubernetes \
  --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \
  --cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --root-ca-file=/opt/kubernetes/ssl/ca.pem \
  --leader-elect=true \
  --v= \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log

Restart=on-failure
RestartSec=

[Install]
WantedBy=multi-user.target

启动Controller Manager

systemctl daemon-reload
systemctl enable kube-controller-manager
systemctl start kube-controller-manager

#查看状态
systemctl status kube-controller-manager
[root@k8s-master ssl]# netstat -lntup|grep kube-controll
tcp               127.0.0.1:         0.0.0.0:*               LISTEN      /kube-controll

部署Kubernetes Scheduler

[root@k8s-master ssl]#  vim /usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
ExecStart=/opt/kubernetes/bin/kube-scheduler \
  --address=127.0.0.1 \
  --master=http://127.0.0.1:8080 \
  --leader-elect=true \
  --v= \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log

Restart=on-failure
RestartSec=

[Install]
WantedBy=multi-user.target

启动Kubernetes Scheduler

systemctl daemon-reload
systemctl enable kube-scheduler
systemctl start kube-scheduler

#查看服务状态
systemctl status kube-scheduler
[root@k8s-master ssl]# netstat -lntup|grep kube-schedule
tcp               127.0.0.1:         0.0.0.0:*               LISTEN      /kube-schedule 

部署kubectl 命令行工具

kubectl是通过API Server来管理k8s集群的，kubectl和API Server之间通信也需要证书认证。kubectl 只在Master管理节点安装，下面来生成证书。

1.准备二进制命令包

[root@k8s-master ssl]# cd /usr/local/src/kubernetes/client/bin
[root@k8s-master bin]# cp kubectl /opt/kubernetes/bin/

2.创建 admin 证书签名请求

[root@k8s-master bin]# cd /usr/local/src/ssl/
[root@k8s-master ssl]# vim admin-csr.json
{
  "CN": "admin",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size":
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "system:masters",
      "OU": "System"
    }
  ]
}

3.生成 admin 证书和私钥

[root@k8s-master ssl]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
>    -ca-key=/opt/kubernetes/ssl/ca-key.pem \
>    -config=/opt/kubernetes/ssl/ca-config.json \
>    -profile=kubernetes admin-csr.json | cfssljson -bare admin
// :: [INFO] generate received request
// :: [INFO] received CSR
// :: [INFO] generating key: rsa-
// :: [INFO] encoded CSR
// :: [INFO] signed certificate with serial number
// :: [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1., from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2. ("Information Requirements").

#会生成4个文件
[root@k8s-master ssl]# ls -l admin*
-rw-r--r--  root root  Nov  : admin.csr
-rw-r--r--  root root   Nov  : admin-csr.json
-rw-------  root root  Nov  : admin-key.pem
-rw-r--r--  root root  Nov  : admin.pem

#移动到ssl证书目录
[root@k8s-master ssl]# mv admin*.pem /opt/kubernetes/ssl/

4.设置集群参数

[root@k8s-master ssl]# kubectl config set-cluster kubernetes \
>    --certificate-authority=/opt/kubernetes/ssl/ca.pem \
>    --embed-certs=true \
>    --server=https://10.0.3.225:6443
Cluster "kubernetes" set.

5.设置客户端认证参数

[root@k8s-master ssl]# kubectl config set-credentials admin \
>    --client-certificate=/opt/kubernetes/ssl/admin.pem \
>    --embed-certs=true \
>    --client-key=/opt/kubernetes/ssl/admin-key.pem
User "admin" set.

6.设置上下文参数

[root@k8s-master ssl]#  kubectl config set-context kubernetes \
>    --cluster=kubernetes \
>    --user=admin
Context "kubernetes" created.

7.设置默认上下文

[root@k8s-master ssl]# kubectl config use-context kubernetes
Switched to context "kubernetes".

#敲了一大堆命令，其实是在家目录.kube/ 生成一个config配置文件，kubectl和API Server通信就要使用到这个文件。 其他节点想要运行kubectl 就要把这个文件拷贝过去
root@k8s-master ~]# cat  .kube/config

8.使用kubectl工具

[root@k8s-master ssl]# kubectl get cs
NAME                 STATUS    MESSAGE              ERROR
controller-manager   Healthy   ok
scheduler            Healthy   ok
etcd-               Healthy   {"health": "true"}
etcd-               Healthy   {"health": "true"}
etcd-               Healthy   {"health": "true"}