python安装二进制k8s高可用 版本1.13.0
一、所有安装包、脚本、脚本说明、下载链接：https://pan.baidu.com/s/1kHaesJJuMQ5cG-O_nvljtg
提取码：kkv6 二、脚本安装说明 1、脚本说明： 本实验为三台master组成高可用，另外一台单独安装node组件,一共四个节点； 脚本里面的几个apiserver地址是写死的，所以部署时候用的ip不一样的话，需要替换； flanneld、coredns 文件里面ip是写死的，所以部署时候用的ip不一样的话，需要替换 /root/ssl/目录下有几个shell脚本里面的一些信息也是写死的，需要替换。 实验采用root进制安装，如果使用其他用户，所有命令前加上sudo； 用到的技术点: shel、python、jinja2、ansible、playbook 如果笔记本配置低的话，最好分开安装，先安装etcd集群，然后重启所有节点，再安装k8s集群。否则cpu会很高。导致到后面安装flananl总是无法启动 测试过5遍，都可以搭建成功的，脚本不要动了，一动就容易出错， 脚本耗时 2小时，如果笔记本配置高，安装完etcd，cpu不高的话，就不需要重启笔记本，那样可能需要1.5个小时 2、项目架构： test1: 192.168.0.91 etcd、kubectl、kube-apiserver、kube-controller-manager、kube-scheduler、haproxy、keepalived ansible python test2: 192.168.0.92 etcd、kubectl、kube-apiserver、kube-controller-manager、kube-scheduler、haproxy、keepalived python test3: 192.168.0.93 etcd、kubectl、kube-apiserver、kube-controller-manager、kube-scheduler、haproxy、keepalived python test4: 192.168.0.94 docker、kubectl工具、kubelet组件、kube-proxy、flannel、coredns python jinja2 3、环境配置 参照本目录中环境配置篇进行操作 说明：环境配置中包含：配置免密登录、安装ansible、关闭防火墙、selinux、swap、修改主机名、配置hosts解析、安装python、pip、重启 4、test1节点创建目录 mkdir -p /k8s/profile/ mkdir -p /server/software/k8s/ mkdir -p /root/ssl/ mkdir -p /script/ 5、需要的文件放到/k8s/profile/目录下 cd /k8s/profile/
rz
unzip profile.zip
chmod +x /k8s/profile/* ls /k8s/profile/ apiserver-to-kubelet.yaml encryption-config.py kube-apiserver.service.template.py.bak
a.py encryption-config-template.py kube-controller-manager.service.py
audit-policy.yaml etcd.service.py kube-controller-manager.service.template.py
bootstrap-copy.sh etcd.service.template.py kube-flannel.yml.j2
bootstrap-copy.sh.bak flanneld.service.py kubelet.config.json.py
bootstrap-kubeconfig.sh flanneld.service.template.py kubelet.config.json.template.py
bootstrap-kubeconfig.sh.bak2 flannel_to_etcd.sh kubelet.service.py
coredns.yaml.j2 haproxy.cfg kubelet.service.template.py
csr-crb.yaml hosts kube-proxy.config.yaml.py
daemon.json k8s.conf kube-proxy.config.yaml.template.py
deploy_coredns.retry keepalived-back.py kube-proxy.service
deploy_coredns.yaml keepalived-backup-template.py kubernetes.conf
deploy_flanneld.retry keepalived-master.py kube-scheduler.service
deploy_flanneld.yaml keepalived-master-template.py profile.zip
deploy_flanneld.yaml.bak kube-apiserver.service.py start-docker.sh
docker.service kube-apiserver.service.template.py token-template.py 6、安装包放到/server/software/k8s/下 cd /server/software/k8s/ rz ls /server/software/k8s/ cfssl-certinfo_linux-amd64 docker-18.03.1-ce.tgz kubernetes README.md
cfssljson_linux-amd64 docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm kubernetes-server-linux-amd64.tar.gz root@192.168.0.94
cfssl_linux-amd64 etcd-v3.3.9-linux-amd64 mk-docker-opts.sh
cni-plugins-amd64-v0.7.1.tgz etcd-v3.3.9-linux-amd64.tar.gz pip-18.0.tar.gz
coredns-1.2.2.tar.gz flannel.tar Python-3.6.5.tgz 7、创建证书所需文件 cd /root/ssl/
rz ssl.zip
unzip ssl.zip ls /root/ssl/ admin.conf ca.csr flanneld.csr kube-controller-manager-csr.json kubernetes-csr.json
admin.csr ca-csr.json flanneld-csr.json kube-proxy.csr kube-scheduler.csr
admin-csr.json etcd.csr k8s.py kube-proxy-csr.json kube-scheduler-csr.json
ca-config.json etcd-csr.json kube-controller-manager.csr kubernetes.csr ssl.zip 8、脚本放到/script/目录下 cd /script/ rz k8s.py ls /script/ k8s.py pip-python.sh 9、给辅助shell脚本授权 chmod +x /k8s/profile/* 9、打开主脚本函数 去掉主脚本里面最后面几行函数注释 10、开始执行 python /script/k8s.py 三、k8s.py脚本内容 [root@test1 script]# cat /script/k8s.py
#!/usr/bin/python
# -*- coding: utf-8 -*- from __future__ import print_function
import os, sys, stat
import shutil
import tarfile
import subprocess # 定义环境变量 # 定义主机名
NODE_NAME = subprocess.check_output(["hostname"], shell=True)
NODE_NAME = str(NODE_NAME.decode('utf8').strip()).strip('b') # 定义主机ip
NODE_IP = subprocess.check_output(["hostname -i | awk '{print $NF}'"], shell=True)
NODE_IP = str(NODE_IP.decode('utf8').strip()).strip('b') #定义key
ENCRYPTION_KEY = subprocess.check_output(["head -c 32 /dev/urandom | base64"], shell=True)
ENCRYPTION_KEY = str(ENCRYPTION_KEY.decode('utf8').strip()).strip('b') SERVICE_CIDR = "10.254.0.0/16"
CLUSTER_CIDR = "172.30.0.0/16"
NODE_PORT_RANGE = "8000-30000"
NODE_IPS = ['192.168.0.91', '192.168.0.92', '192.168.0.93']
NODE_NAMES = ['test1', 'test2', 'test3','test4']
MASTER_VIP = "192.168.0.235"
KUBE_APISERVER = "https://192.168.0.235:8443"
VIP_IF = "ens33"
ETCD_ENDPOINTS = "https://192.168.0.91:2379,https://192.168.0.92:2379,https://192.168.0.93:2379"
ETCD_NODES = "test1=https://192.168.0.91:2380,test2=https://192.168.0.92:2380,test3=https://192.168.0.93:2380"
FLANNEL_ETCD_PREFIX = "/kubernetes/network/"
CLUSTER_KUBERNETES_SVC_IP = "10.254.0.1"
CLUSTER_DNS_SVC_IP = "10.254.0.2"
CLUSTER_DNS_DOMAIN = "cluster.local."
IFACE="ens33" def create_dir_kernel_parameters():
print("创建目录、配置内核参数、安装依赖包")
#所有节点创建目录
subprocess.call(["time ansible k8s -m file -a 'path=/k8s/profile/ state=directory mode=0777'"], shell=True)
subprocess.call(["time ansible k8s -m file -a 'path=/server/software/k8s/ state=directory mode=0777'"], shell=True)
subprocess.call(["time ansible k8s -m file -a 'path=/root/ssl/ state=directory mode=0777'"], shell=True)
subprocess.call(["time ansible k8s -m file -a 'path=/script/ state=directory mode=0777'"], shell=True)
#配置内核参数
subprocess.call(["ansible k8s -m shell -a 'iptables -P FORWARD ACCEPT'"], shell=True)
subprocess.call(["time ansible k8s -m copy -a 'src=/k8s/profile/k8s.conf dest=/etc/sysctl.d/k8s.conf force=yes'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'sysctl --system'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'modprobe ip_vs'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'modprobe ip_vs_rr'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'modprobe ip_vs_wrr'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'modprobe ip_vs_sh'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'modprobe nf_conntrack_ipv4'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'lsmod | grep ip_vs'"], shell=True)
#安装依赖包， 其实这一步只是配置node节点的kube-proxy步骤需要用到
subprocess.call(["ansible k8s -m shell -a 'yum install -y conntrack ipvsadm ipset jq sysstat curl iptables libseccomp'"], shell=True) def install_cfssl():
print("安装 cfssl 工具集")
subprocess.call(["ansible k8s -m file -a 'path=/server/software/k8s/ state=directory mode=0777'"], shell=True)
os.chdir('/server/software/k8s/')
subprocess.call(["ansible test1 -m copy -a 'src=/server/software/k8s/cfssl-certinfo_linux-amd64 dest=/usr/local/bin/cfssl-certinfo force=yes'"], shell=True)
subprocess.call(["ansible test1 -m copy -a 'src=/server/software/k8s/cfssl_linux-amd64 dest=/usr/local/bin/cfssl'"], shell=True)
subprocess.call(["ansible test1 -m copy -a 'src=/server/software/k8s/cfssljson_linux-amd64 dest=/usr/local/bin/cfssljson'"], shell=True)
os.chdir('/usr/local/bin/')
os.chmod("cfssl-certinfo", stat.S_IXOTH)
os.chmod("cfssl", stat.S_IXOTH)
os.chmod("cfssljson", stat.S_IXOTH)
print("successful") def create_root_ca():
print("创建CA根证书")
os.chdir('/root/ssl/')
subprocess.call(["time ansible k8s -m file -a 'path=/root/ssl/ state=directory mode=0777'"], shell=True)
subprocess.call(["cfssl gencert -initca ca-csr.json | cfssljson -bare ca"], shell=True)
#创建用户
subprocess.call(["time ansible k8s -m shell -a 'useradd k8s'"],shell=True)
#创建目录
subprocess.call(["time ansible k8s -m shell -a 'mkdir -p /etc/kubernetes/cert/ && chown -R k8s /etc/kubernetes'"],shell=True)
subprocess.call(["time ansible k8s -m copy -a 'src=/root/ssl/ dest=/etc/kubernetes/cert/ force=yes'"], shell=True)
print("successful") def distribute_kubectl():
print("分发 所有 二进制文件、安装kubectl ")
os.chdir('/server/software/k8s/')
shutil.unpack_archive('kubernetes-server-linux-amd64.tar.gz')
subprocess.call(["time ansible k8s -m file -a 'path=/opt/k8s/bin/ state=directory mode=0777'"], shell=True)
subprocess.call(["time ansible k8s -m copy -a 'src=/server/software/k8s/kubernetes/server/bin/ dest=/opt/k8s/bin/ force=yes'"],shell=True)
subprocess.call(["chmod +x /opt/k8s/bin/*"], shell=True)
subprocess.call(["time ansible k8s -m copy -a 'src=/opt/k8s/bin/kubectl dest=/usr/local/bin/ force=yes'"], shell=True)
subprocess.call(["time ansible k8s -m shell -a 'chmod +x /usr/local/bin/kubectl'"], shell=True)
print("successful") def create_admin_ca():
print("创建 admin 证书和私钥 ")
os.chdir('/root/ssl/')
subprocess.call(["ansible k8s -m shell -a 'chmod +x /etc/kubernetes/cert/*'"], shell=True)
subprocess.call(["cfssl gencert -ca=/etc/kubernetes/cert/ca.pem -ca-key=/etc/kubernetes/cert/ca-key.pem -config=/etc/kubernetes/cert/ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin"],shell=True)
subprocess.call(["ansible k8s -m copy -a 'src=/root/ssl/admin.pem dest=/etc/kubernetes/cert/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m copy -a 'src=/root/ssl/admin-key.pem dest=/etc/kubernetes/cert/ force=yes'"], shell=True)
#给证书授权
subprocess.call(["ansible k8s -m shell -a 'chmod +x /etc/kubernetes/cert/*'"], shell=True)
print("successful") def create_admin_kubeconfig():
print("创建admin kubeconfig 文件")
os.chdir('/root/ssl/')
subprocess.call(["kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/cert/ca.pem --embed-certs=true --server=https://192.168.0.235:8443 --kubeconfig=admin.conf"],shell=True)
subprocess.call(["kubectl config set-credentials admin --client-certificate=admin.pem --client-key=admin-key.pem --embed-certs=true --kubeconfig=admin.conf"],shell=True)
subprocess.call(["kubectl config set-context kubernetes --cluster=kubernetes --user=admin --kubeconfig=admin.conf"],shell=True)
subprocess.call(["kubectl config use-context kubernetes --kubeconfig=admin.conf"],shell=True)
subprocess.call(["ansible k8s -m file -a 'path=/root/.kube/ state=directory mode=0777'"], shell=True)
subprocess.call(["ansible k8s -m copy -a 'src=/root/ssl/admin.conf dest=/root/.kube/config force=yes'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'chmod +x /root/.kube/*'"], shell=True)
print("创建admin kubeconfig 文件完成") def install_etcd_cluster():
print("部署 etcd 集群")
# 解压和分发 etcd 二进制文件
os.chdir('/server/software/k8s/')
shutil.unpack_archive('etcd-v3.3.9-linux-amd64.tar.gz')
subprocess.call(["time ansible k8s -m copy -a 'src=/server/software/k8s/etcd-v3.3.9-linux-amd64/ dest=/opt/k8s/bin/ force=yes'"],shell=True)
subprocess.call(["time ansible k8s -m shell -a 'chmod +x /opt/k8s/bin/*'"],shell=True)
subprocess.call(["time ansible k8s -m shell -a 'ln -s /opt/k8s/bin/etcdctl /usr/bin/etcdctl'"],shell=True)
# 创建 etcd 证书和私钥
os.chdir('/root/ssl/')
subprocess.call(["cfssl gencert -ca=/etc/kubernetes/cert/ca.pem -ca-key=/etc/kubernetes/cert/ca-key.pem -config=/etc/kubernetes/cert/ca-config.json -profile=kubernetes etcd-csr.json | cfssljson -bare etcd"],shell=True)
subprocess.call(["time ansible k8s -m file -a 'path=/etc/etcd/cert/ state=directory mode=0777'"], shell=True)
subprocess.call(["time ansible k8s -m copy -a 'src=/root/ssl/ dest=/etc/etcd/cert/ force=yes'"], shell=True)
# 创建 etcd.service文件
subprocess.call(["time ansible k8s -m file -a 'path=/k8s/profile/ state=directory mode=0777'"], shell=True)
subprocess.call(["time ansible k8s -m copy -a 'src=/k8s/profile/ dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["time ansible k8s -m shell -a 'python /k8s/profile/etcd.service.py'"],shell=True)
#创建 etcd 数据目录
subprocess.call(["time ansible k8s -m file -a 'path=/var/lib/etcd/ state=directory mode=0777'"], shell=True)
subprocess.call(["time ansible k8s -m shell -a 'chmod +x /var/lib/etcd'"],shell=True)
#启动 etcd 服务
subprocess.call(["time ansible k8s -m shell -a 'systemctl daemon-reload'"],shell=True)
subprocess.call(["time ansible k8s -m service -a 'name=etcd state=started'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=etcd enabled=yes'"],shell=True)
#检查启动结果
subprocess.call(["time ansible k8s -m shell -a 'systemctl status etcd'"],shell=True)
#验证服务状态
subprocess.call(["/opt/k8s/bin/etcdctl --ca-file=/etc/kubernetes/cert/ca.pem --cert-file /etc/etcd/cert/etcd.pem --key-file /etc/etcd/cert/etcd-key.pem cluster-health"],shell=True)
#说明：如果启动失败就把所有节点关机重启就即可解决 def install_haproxy():
print("安装haproxy")
#安装软件
subprocess.call(["ansible k8s -m shell -a 'yum install -y haproxy'"],shell=True)
#配置和下发 haproxy 配置文件
subprocess.call(["ansible k8s -m copy -a 'src=/k8s/profile/haproxy.cfg dest=/etc/haproxy force=yes'"], shell=True)
#启动haproxy
subprocess.call(["ansible k8s -m shell -a 'systemctl daemon-reload'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=haproxy state=started'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=haproxy enabled=yes'"],shell=True)
#检查启动结果
subprocess.call(["time ansible k8s -m shell -a 'systemctl status haproxy'"],shell=True) def install_keepalived():
print("安装keepalived")
#安装软件
subprocess.call(["ansible k8s -m shell -a 'yum install -y keepalived'"],shell=True)
#配置和下发 keepalived 配置文件
subprocess.call(["ansible k8s -m copy -a 'src=/k8s/profile/ dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible test1 -m shell -a 'python /k8s/profile/keepalived-master.py'"],shell=True)
subprocess.call(["ansible test0 -m shell -a 'python /k8s/profile/keepalived-back.py'"],shell=True)
#起动keepalived 服务
subprocess.call(["ansible k8s -m shell -a 'systemctl daemon-reload'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=keepalived state=started'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=keepalived enabled=yes'"],shell=True)
#检查启动结果
subprocess.call(["time ansible k8s -m shell -a 'systemctl status keepalived'"],shell=True)
#测试是否能ping通vip
subprocess.call(["time ansible k8s -m shell -a 'ping -c 1 192.168.0.235'"],shell=True)
#查看 haproxy 状态页面 http://192.168.0.235:10080/status；账号密码：admin/123456 def kube_apiserver():
print("部署 kube-apiserver 组件")
#创建 kubernetes 证书和私钥
os.chdir('/root/ssl/')
subprocess.call(["cfssl gencert -ca=/etc/kubernetes/cert/ca.pem -ca-key=/etc/kubernetes/cert/ca-key.pem -config=/etc/kubernetes/cert/ca-config.json -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes"], shell=True)
#生产service account key
subprocess.call(["ansible test1 -m shell -a 'openssl genrsa -out /root/ssl/sa.key 2048'"], shell=True)
subprocess.call(["ansible test1 -m shell -a 'openssl rsa -in /root/ssl/sa.key -pubout -out /root/ssl/sa.pub'"], shell=True)
#将生成的证书、私钥、ervice account key 分发到所有节点
subprocess.call(["ansible k8s -m copy -a 'src=/root/ssl/ dest=/etc/kubernetes/cert/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'chmod +x /etc/kubernetes/cert/'"], shell=True)
#创建、分发加密配置文件
subprocess.call(["ansible k8s -m shell -a 'python /k8s/profile/encryption-config.py'"],shell=True)
subprocess.call(["ansible k8s -m copy -a 'src=/etc/kubernetes/encryption-config.yaml dest=/etc/kubernetes force=yes'"], shell=True)
#创建、分发kube-apiserver文件
subprocess.call(["ansible k8s -m copy -a 'src=/k8s/profile/kube-apiserver.service.py dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m copy -a 'src=/k8s/profile/kube-apiserver.service.template.py dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'python /k8s/profile/kube-apiserver.service.py'"],shell=True)
#起动kube-apiserver 服务
subprocess.call(["ansible k8s -m shell -a 'systemctl daemon-reload'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=kube-apiserver state=started'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=kube-apiserver enabled=yes'"],shell=True)
#检查 kube-apiserver服务 运行状态
subprocess.call(["time ansible k8s -m shell -a 'systemctl status kube-apiserver'"],shell=True)
#检查集群运行状态
subprocess.call(["ansible k8s -m shell -a 'kubectl cluster-info'"],shell=True)
subprocess.call(["ansible k8s -m shell -a 'kubectl get componentstatuses'"],shell=True)
#授予 kubernetes 证书访问 kubelet API 的权限, 这句话感觉没什么用，到最后还是出现Forbidden (user=kubernetes, verb=create, resource=nodes, subresource=proxy)
#所以在安装kubelet时候采取了绝招。
subprocess.call(["ansible k8s -m shell -a 'kubectl create clusterrolebinding kube-apiserver:kubelet-apis --clusterrole=system:kubelet-api-admin --user kubernete'"],shell=True)
print("上面报错忽略，不要乱改脚本了，之前这里只是改成了test1，集群就无法启动了")
subprocess.call(["ansible k8s -m shell -a 'sleep 60s'"], shell=True) def kube_controller_manager():
print("部署高可用 kube-controller-manager 集群")
#创建 kubernetes 证书和私钥
os.chdir('/root/ssl/')
subprocess.call(["cfssl gencert -ca=/etc/kubernetes/cert/ca.pem -ca-key=/etc/kubernetes/cert/ca-key.pem -config=/etc/kubernetes/cert/ca-config.json -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager"], shell=True)
#将生成的证书和私钥分发到所有节点
subprocess.call(["ansible k8s -m copy -a 'src=/root/ssl/ dest=/etc/kubernetes/cert/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'chmod +x /etc/kubernetes/cert/'"], shell=True)
#创建和分发 kubeconfig 文件
subprocess.call(["kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/cert/ca.pem --embed-certs=true --server=https://192.168.0.235:8443 --kubeconfig=kube-controller-manager.kubeconfig"], shell=True)
subprocess.call(["kubectl config set-credentials system:kube-controller-manager --client-certificate=kube-controller-manager.pem --client-key=kube-controller-manager-key.pem --embed-certs=true --kubeconfig=kube-controller-manager.kubeconfig"], shell=True)
subprocess.call(["kubectl config set-context system:kube-controller-manager --cluster=kubernetes --user=system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig"], shell=True)
subprocess.call(["kubectl config use-context system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig"], shell=True)
subprocess.call(["ansible k8s -m copy -a 'src=/root/ssl/kube-controller-manager.kubeconfig dest=/etc/kubernetes/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'chmod +x /etc/kubernetes/*'"], shell=True)
#创建、分发kube-controller-manager.service
subprocess.call(["ansible k8s -m copy -a 'src=/k8s/profile/kube-controller-manager.service.template.py dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m copy -a 'src=/k8s/profile/kube-controller-manager.service.py dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'python /k8s/profile/kube-controller-manager.service.py'"],shell=True)
#创建日志目录
subprocess.call(["ansible k8s -m file -a 'path=/var/log/kubernetes/ state=directory mode=0777'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'chmod +x /var/log/kubernetes/'"], shell=True)
#启动 kube-controller-manager 服务
subprocess.call(["ansible k8s -m shell -a 'systemctl daemon-reload'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=kube-controller-manager state=started'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=kube-controller-manager enabled=yes'"],shell=True)
#检查 kube-controller-manager 运行状态
subprocess.call(["time ansible k8s -m shell -a 'systemctl status kube-controller-manager'"],shell=True)
#查看leader节点
subprocess.call(["kubectl get endpoints kube-controller-manager --namespace=kube-system -o yaml"],shell=True) def install_kube_scheduler():
print("部署高可用 kube-scheduler 集群")
#创建 kube-scheduler 证书和私钥
os.chdir('/root/ssl/')
subprocess.call(["cfssl gencert -ca=/etc/kubernetes/cert/ca.pem -ca-key=/etc/kubernetes/cert/ca-key.pem -config=/etc/kubernetes/cert/ca-config.json -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler"], shell=True)
#将生成的证书和私钥分发到所有节点
subprocess.call(["ansible k8s -m copy -a 'src=/root/ssl/kube-scheduler.pem dest=/etc/kubernetes/cert/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m copy -a 'src=/root/ssl/kube-scheduler-key.pem dest=/etc/kubernetes/cert/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'chmod +x /etc/kubernetes/cert/'"], shell=True)
#创建和分发 kubeconfig 文件
subprocess.call(["kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/cert/ca.pem --embed-certs=true --server=https://192.168.0.235:8443 --kubeconfig=kube-scheduler.kubeconfig"], shell=True)
subprocess.call(["kubectl config set-credentials system:kube-scheduler --client-certificate=kube-scheduler.pem --client-key=kube-scheduler-key.pem --embed-certs=true --kubeconfig=kube-scheduler.kubeconfig"], shell=True)
subprocess.call(["kubectl config set-context system:kube-scheduler --cluster=kubernetes --user=system:kube-scheduler --kubeconfig=kube-scheduler.kubeconfig"], shell=True)
subprocess.call(["kubectl config use-context system:kube-scheduler --kubeconfig=kube-scheduler.kubeconfig"], shell=True)
subprocess.call(["ansible k8s -m copy -a 'src=/root/ssl/kube-scheduler.kubeconfig dest=/etc/kubernetes/ force=yes'"], shell=True)
subprocess.call(["ansible k8s -m shell -a 'chmod +x /etc/kubernetes/*'"], shell=True)
#创建和分发kube-scheduler.service
subprocess.call(["ansible k8s -m copy -a 'src=/k8s/profile/kube-scheduler.service dest=/etc/systemd/system/ force=yes'"], shell=True)
#启动kube-scheduler 服务
subprocess.call(["ansible k8s -m shell -a 'systemctl daemon-reload'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=kube-scheduler state=started'"],shell=True)
subprocess.call(["ansible k8s -m service -a 'name=kube-scheduler enabled=yes'"],shell=True)
#检查 kube-scheduler 运行状态
subprocess.call(["time ansible k8s -m shell -a 'systemctl status kube-scheduler'"],shell=True)
#查看leader节点
subprocess.call(["kubectl get endpoints kube-scheduler --namespace=kube-system -o yaml"],shell=True) def environmental_configuration():
print("下面开始单独安装worker节点，worker节点主机名：test4")
print("配置环境")
#所有节点创建目录
subprocess.call(["time ansible test4 -m file -a 'path=/k8s/profile/ state=directory mode=0777'"], shell=True)
subprocess.call(["time ansible test4 -m file -a 'path=/server/software/k8s/ state=directory mode=0777'"], shell=True)
subprocess.call(["time ansible test4 -m file -a 'path=/root/ssl/ state=directory mode=0777'"], shell=True)
subprocess.call(["time ansible test4 -m file -a 'path=/script/ state=directory mode=0777'"], shell=True)
#加载ipvs内核参数，其实这一步在安装kube-proxy那一步时执行即可
subprocess.call(["ansible test4 -m shell -a 'iptables -P FORWARD ACCEPT'"], shell=True)
subprocess.call(["time ansible test4 -m copy -a 'src=/k8s/profile/k8s.conf dest=/etc/sysctl.d/k8s.conf force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'sysctl --system'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'modprobe ip_vs'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'modprobe ip_vs_rr'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'modprobe ip_vs_wrr'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'modprobe ip_vs_sh'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'modprobe nf_conntrack_ipv4'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'lsmod | grep ip_vs'"], shell=True)
#安装依赖包， 其实这一步只是配置node节点的kube-proxy步骤需要用到
subprocess.call(["ansible test4 -m shell -a 'yum install -y conntrack ipvsadm ipset jq sysstat curl iptables libseccomp'"], shell=True)
#创建k8s群组
subprocess.call(["ansible test4 -m group -a 'name=k8s state=present'"], shell=True)
#创建k8s用户
subprocess.call(["ansible test4 -m user -a 'name=k8s group=k8s'"], shell=True) def install_docker():
print("部署docker")
#移除自带安装包
subprocess.call(["ansible test4 -m shell -a 'yum remove -y docker-ce docker-ce-selinux container-selinux'"],shell=True)
#下发docker安装包
subprocess.call(["ansible test4 -m copy -a 'src=/server/software/k8s/docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm dest=/server/software/k8s/ force=yes'"], shell=True)
#安装docker
subprocess.call(["ansible test4 -m shell -a 'chdir=/server/software/k8s/ yum install -y docker-ce-*.rpm'"],shell=True)
#配置docker加速
subprocess.call(["ansible test4 -m copy -a 'src=/k8s/profile/daemon.json dest=/etc/docker/ force=yes'"], shell=True)
#启动docker
subprocess.call(["ansible test4 -m shell -a 'systemctl start docker'"],shell=True)
subprocess.call(["ansible test4 -m shell -a 'systemctl enable docker'"],shell=True) def install_kubelet():
print("部署 kubelet 组件")
os.chdir('/root/ssl/')
#创建工作目录和日志目录
subprocess.call(["ansible test4 -m file -a 'path=/var/lib/kubelet/ state=directory mode=0777'"], shell=True)
subprocess.call(["ansible test4 -m file -a 'path=/var/log/kubernetes/ state=directory mode=0777'"], shell=True)
#配置admin-config
subprocess.call(["ansible test4 -m file -a 'path=/root/.kube/ state=directory mode=0777'"], shell=True)
subprocess.call(["ansible test4 -m copy -a 'src=/root/ssl/admin.conf dest=/root/.kube/config force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'chmod +x /root/.kube/*'"], shell=True)
#配置kubectl工具
subprocess.call(["ansible test4 -m copy -a 'src=/opt/k8s/bin/kubectl dest=/usr/local/bin/ force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'chmod +x /usr/local/bin/kubectl'"], shell=True)
#配置kubeadm工具， 注意：在test1上操作
subprocess.call(["ansible test1 -m copy -a 'src=/opt/k8s/bin/kubeadm dest=/usr/local/bin/ force=yes'"], shell=True)
subprocess.call(["ansible test1 -m shell -a 'chmod +x /usr/local/bin/kubeadm'"], shell=True)
#创建 kubelet bootstrap kubeconfig 文件，包含创建token
subprocess.call(["ansible test1 -m shell -a 'chmod +x /k8s/profile/bootstrap-kubeconfig.sh'"], shell=True)
subprocess.call(["ansible test1 -m shell -a 'sh /k8s/profile/bootstrap-kubeconfig.sh'"], shell=True)
#分发kubelet bootstrap kubeconfig 文件到所有节点
subprocess.call(["ansible test1 -m shell -a 'chmod +x /k8s/profile/bootstrap-copy.sh'"], shell=True)
subprocess.call(["ansible test1 -m shell -a 'sh /k8s/profile/bootstrap-copy.sh'"], shell=True)
print("上面报错忽略，没有影响")
subprocess.call(["ansible test4 -m shell -a 'sleep 30s'"],shell=True)
#配置 bootstrap RBAC 权限
subprocess.call(["kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --group=system:bootstrappers"],shell=True)
#查看 kubeadm 为各节点创建的 token；之前好好的，后来查看报错
#subprocess.call(["ansible test1 -m shell -a 'kubeadm token list --kubeconfig ~/.kube/config'"], shell=True)
#分发ca证书到worker 节点
subprocess.call(["ansible test4 -m copy -a 'src=/root/ssl/ca.pem dest=/etc/kubernetes/cert/ force=yes'"], shell=True)
#创建 kubelet 参数配置文件
subprocess.call(["ansible test4 -m copy -a 'src=/k8s/profile/kubelet.config.json.template.py dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible test4 -m copy -a 'src=/k8s/profile/kubelet.config.json.py dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'python /k8s/profile/kubelet.config.json.py'"],shell=True)
#分发kubelet二进制文件
subprocess.call(["time ansible test4 -m copy -a 'src=/opt/k8s/bin/kubelet dest=/opt/k8s/bin/ force=yes'"], shell=True)
subprocess.call(["time ansible test4 -m shell -a 'chmod +x /opt/k8s/bin/kubelet'"], shell=True)
#创建和分发kubelet启动文件
subprocess.call(["ansible test4 -m copy -a 'src=/k8s/profile/kubelet.service.template.py dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible test4 -m copy -a 'src=/k8s/profile/kubelet.service.py dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'python /k8s/profile/kubelet.service.py'"],shell=True)
#给kunelet授权操作pod权限，包括进入pod等操作，真正解决 Forbidden (user=kubernetes,）问题
subprocess.call(["ansible test4 -m copy -a 'src=/k8s/profile/apiserver-to-kubelet.yaml dest=/k8s/profile/apiserver-to-kubelet.yaml force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'kubectl create -f /k8s/profile/apiserver-to-kubelet.yaml'"],shell=True)
#启动 kubelet 服务
subprocess.call(["ansible test4 -m shell -a 'systemctl daemon-reload'"],shell=True)
subprocess.call(["ansible test4 -m service -a 'name=kubelet state=started'"],shell=True)
subprocess.call(["ansible test4 -m service -a 'name=kubelet enabled=yes'"],shell=True)
#检查启动结果
subprocess.call(["ansible test4 -m shell -a 'systemctl status kubelet'"],shell=True)
#设置集群角色
test4=subprocess.check_output(["kubectl get nodes | grep test4 | awk '{print $1}'"], shell=True)
test4=test4.decode('utf8').strip()
subprocess.call(['kubectl','label','nodes',test4,'node-role.kubernetes.io/node='])
print("查看csr,如果是Approved,Issued状态，下面这个报错忽略: No resources found error: resource name may not be empty")
subprocess.call(["ansible test4 -m shell -a 'kubectl get csr'"],shell=True)
subprocess.call(["ansible test4 -m shell -a 'sleep 60s'"],shell=True) def approve_csr():
print("自动 approve CSR 请求")
subprocess.call(["time ansible test4 -m copy -a 'src=/k8s/profile/csr-crb.yaml dest=/k8s/profile/csr-crb.yaml force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'kubectl apply -f /k8s/profile/csr-crb.yaml'"],shell=True)
print("No resources found error: resource name may not be empty 这个报错忽略，不影响的，")
subprocess.call(["ansible test4 -m shell -a 'sleep 30s'"],shell=True)
#等待一段时间(1-10 分钟)，所有节点的 CSR 都被自动 approve；看了下时间，这次刚好等了10分钟
subprocess.call(["ansible test4 -m shell -a 'sleep 300s'"],shell=True)
#查看csr
subprocess.call(["ansible test4 -m shell -a 'kubectl get csr'"],shell=True)
#所有节点均 ready
subprocess.call(["ansible test4 -m shell -a 'kubectl get nodes'"],shell=True)
#查看kube-controller-manager 为各 node 生成了 kubeconfig 文件和公私钥
subprocess.call(["ansible test4 -m shell -a 'ls -l /etc/kubernetes/kubelet.kubeconfig'"],shell=True)
subprocess.call(["ansible test4 -m shell -a 'ls -l /etc/kubernetes/cert/|grep kubelet'"],shell=True)
#查看kubelet 提供的 API 接口
subprocess.call(["ansible test4 -m shell -a 'yum install net-tools -y'"],shell=True)
subprocess.call(["ansible test4 -m shell -a 'netstat -lnpt|grep kubelet'"],shell=True) def install_kube_proxy():
print("部署 kube-proxy 组件")
#创建工作目录和日志目录
subprocess.call(["ansible test4 -m file -a 'path=/var/lib/kube-proxy/ state=directory mode=0777'"], shell=True)
subprocess.call(["ansible test4 -m file -a 'path=/var/log/kubernetes/ state=directory mode=0777'"], shell=True)
subprocess.call(["ansible test4 -m file -a 'path=/var/log/kubernetes/ owner=k8s group=k8s mode=0777'"], shell=True)
#安装依赖包
subprocess.call(["ansible test4 -m shell -a 'yum install -y conntrack ipvsadm ipset jq sysstat curl iptables libseccomp'"], shell=True)
#加载 ip_vs 内核模块
subprocess.call(["ansible test4 -m shell -a 'iptables -P FORWARD ACCEPT'"], shell=True)
subprocess.call(["time ansible test4 -m copy -a 'src=/k8s/profile/k8s.conf dest=/etc/sysctl.d/k8s.conf force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'sysctl --system'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'modprobe ip_vs'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'modprobe ip_vs_rr'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'modprobe ip_vs_wrr'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'modprobe ip_vs_sh'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'modprobe nf_conntrack_ipv4'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'lsmod | grep ip_vs'"], shell=True)
#创建 kube-proxy 证书和私钥
os.chdir('/root/ssl/')
subprocess.call(["cfssl gencert -ca=/etc/kubernetes/cert/ca.pem -ca-key=/etc/kubernetes/cert/ca-key.pem -config=/etc/kubernetes/cert/ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy"], shell=True)
#将生成的证书和私钥分发到test4节点
subprocess.call(["ansible test4 -m copy -a 'src=/root/ssl/kube-proxy.pem dest=/etc/kubernetes/cert/ force=yes'"], shell=True)
subprocess.call(["ansible test4 -m copy -a 'src=/root/ssl/kube-proxy-key.pem dest=/etc/kubernetes/cert/ force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'chmod +x /etc/kubernetes/cert/'"], shell=True)
#创建和分发 kubeconfig 文件
subprocess.call(["kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/cert/ca.pem --embed-certs=true --server=https://192.168.0.235:8443 --kubeconfig=kube-proxy.kubeconfig"], shell=True)
subprocess.call(["kubectl config set-credentials system:kube-proxy --client-certificate=kube-proxy.pem --client-key=kube-proxy-key.pem --embed-certs=true --kubeconfig=kube-proxy.kubeconfig"], shell=True)
subprocess.call(["kubectl config set-context system:kube-proxy --cluster=kubernetes --user=system:kube-proxy --kubeconfig=kube-proxy.kubeconfig"], shell=True)
subprocess.call(["kubectl config use-context system:kube-proxy --kubeconfig=kube-proxy.kubeconfig"], shell=True)
subprocess.call(["ansible test4 -m copy -a 'src=/root/ssl/kube-proxy.kubeconfig dest=/etc/kubernetes/ force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'chmod +x /etc/kubernetes/*'"], shell=True)
#创建 kube-proxy 配置文件
subprocess.call(["ansible test4 -m copy -a 'src=/k8s/profile/kube-proxy.config.yaml.template.py dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible test4 -m copy -a 'src=/k8s/profile/kube-proxy.config.yaml.py dest=/k8s/profile/ force=yes'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'python /k8s/profile/kube-proxy.config.yaml.py'"],shell=True)
#分发kube-proxy二进制文件
subprocess.call(["time ansible test4 -m copy -a 'src=/opt/k8s/bin/kube-proxy dest=/opt/k8s/bin/ force=yes'"], shell=True)
subprocess.call(["time ansible test4 -m shell -a 'chmod +x /opt/k8s/bin/kube-proxy'"], shell=True)
#创建和分发kube-proxy.service
subprocess.call(["ansible test4 -m copy -a 'src=/k8s/profile/kube-proxy.service dest=/etc/systemd/system/kube-proxy.service force=yes'"], shell=True)
#启动kube-proxy 服务
subprocess.call(["ansible test4 -m shell -a 'systemctl daemon-reload'"],shell=True)
subprocess.call(["ansible test4 -m service -a 'name=kube-proxy state=started'"],shell=True)
subprocess.call(["ansible test4 -m service -a 'name=kube-proxy enabled=yes'"],shell=True)
#检查 kube-proxy 运行状态
subprocess.call(["time ansible test4 -m shell -a 'systemctl status kube-proxy'"],shell=True) def install_flanneld():
print("部署 flannel 网络")
#关闭防火墙
subprocess.call(["ansible test4 -m service -a 'name=firewalld state=stopped'"],shell=True)
subprocess.call(["time ansible test4 -m shell -a 'systemctl disable firewalld'"],shell=True)
#安装jinja2模块
subprocess.call(["ansible test4 -m shell -a 'pip3 install jinja2'"], shell=True)
#通过playbook安装flannel，在playbook中定义了通过kubectl安装flanneld;playbook中定义了安装在test4节点上
subprocess.call(["ansible-playbook /k8s/profile/deploy_flanneld.yaml"], shell=True)
#等（1-10）分钟查看flannel pod
subprocess.call(["ansible test4 -m shell -a 'sleep 100s'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'kubectl get pod -n kube-system'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'kubectl get nodes'"], shell=True) def install_coredns():
print("安装coredns")
print("下面报错忽略，没有影响。这个错误: No package jq available.Error: Nothing to donon-zero return code")
subprocess.call(["ansible test4 -m shell -a 'yum install -y jq 2> /dev/null'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'sleep 30s'"], shell=True)
#通过playbook安装coredns，在playbook中定义了通过kubectl安装coredns;playbook中定义了安装在test4节点上
subprocess.call(["ansible-playbook /k8s/profile/deploy_coredns.yaml"], shell=True)
#等（1-10）分钟查看coredns pod
subprocess.call(["ansible test4 -m shell -a 'sleep 60s'"], shell=True)
subprocess.call(["ansible test4 -m shell -a 'kubectl get pod -n kube-system'"], shell=True)
#测试coredns请参照https://www.cnblogs.com/effortsing/p/10312081.html 最后部分 def func_list():
#install_cfssl()
#create_root_ca()
#distribute_kubectl()
#create_admin_ca()
#create_admin_kubeconfig()
#install_etcd_cluster()
#install_haproxy()
#install_keepalived()
#kube_apiserver()
#kube_controller_manager()
#install_kube_scheduler()
#下面开始单独安装worker节点，worker节点地址：192.168.0.94
#environmental_configuration()
#install_docker()
#install_kubelet()
#approve_csr()
#install_kube_proxy()
#install_flanneld()
#install_coredns()
def main():
func_list()
if __name__ == '__main__':
main()
python安装二进制k8s高可用 版本1.13.0的更多相关文章
- python安装二进制k8s 1.11.0 一个master、一个node 查看node节点是主机名---apiserver无法启动，后来改了脚本应该可以
一.脚本说明: 本实验中master.node.etcd都是单体. 安装顺序为:先安装test1节点主要组件,然后开始安装test2节点,最后回头把test1节点加入集群中,这样做目的是理解以后扩容都 ...
- .Net Core2.1 秒杀项目一步步实现CI/CD(Centos7.2)系列一:k8s高可用集群搭建总结以及部署API到k8s
前言:本系列博客又更新了,是博主研究很长时间,亲自动手实践过后的心得,k8s集群是购买了5台阿里云服务器部署的,这个集群差不多搞了一周时间,关于k8s的知识点,我也是刚入门,这方面的知识建议参考博客园 ...
- Rancher安装多节点高可用(HA)
Rancher版本:Rancher v1.0.1 基本配置需求 多节点的HA配置请参照单节点需求 节点需要开放的端口 全局访问:TCP 端口22,80,443,18080(可选:用于在集群启动前 查看 ...
- Centos7 安装keepalived实现高可用
场景:尝试安装keepalived实现高可用,进而在suse环境中部署. 测试过程需要配合Nginx的相关知识:Centos7 Nginx安装 1 安装过程 问题 !!! OpenSSL is not ...
- 一、k8s介绍（第一章、k8s高可用集群安装）
作者:北京小远 出处:http://www.cnblogs.com/bj-xy/ 参考课程:Kubernetes全栈架构师(电脑端购买优惠) 文档禁止转载,转载需标明出处,否则保留追究法律责任的权利! ...
- 全部二进制脚本高可用--只有docker启动未成功
[root@test1 script]# cat k8s-docker-binary-py #!/usr/bin/python # -*- coding: utf-8 -*- from __futur ...
- kubeadm实现k8s高可用集群环境部署与配置
高可用架构 k8s集群的高可用实际是k8s各核心组件的高可用,这里使用主备模式,架构如下: 主备模式高可用架构说明: 核心组件 高可用模式 高可用实现方式 apiserver 主备 keepalive ...
- 【葵花宝典】lvs+keepalived部署kubernetes(k8s)高可用集群
一.部署环境 1.1 主机列表 主机名 Centos版本 ip docker version flannel version Keepalived version 主机配置 备注 lvs-keepal ...
- Centos7使用kubeadm 安装多主高可用kubernets:v.1.11集群
实验环境介绍: 本次实验环境是5个节点 3台master 2台node节点: k8smaster01 192.168.111.128 软件:etcd k8smaster haproxy keepali ...
随机推荐
- The fastest MySQL Sandbox setup ever!
MySQL-Sandbox 3.1.10 introduces a new utility, different from anything I have put before in the MySQ ...
- es中的相关知识一(基本知识和id的定义)
一.es中文档的元数据包括: 1._index: 索引(index)类似于关系型数据库里的数据库(database),事实上,我们的数据被存储和索引在分片(shards)中,索引知识把一个或多个分片分 ...
- Codeforces 1187 G - Gang Up
G - Gang Up 思路: 每个点按时间拆点建边,然后跑最小费用流 一次走的人不能太多,假设每次走的人为k (k*k-(k-1)*(k-1))*d <= c+d k <= 24 代码: ...
- Appium&Java自动化实现移动端几种典型动作
一.Appium4.0 Pinch&Zoom /* * @FileName Pinch_Zoom: Pinch_Zoom * @author davieyang * @create 2018- ...
- 按照教程自动安装RFNoC时.在使用pip安装pybombs时出现报错,解决办法
$ sudo apt-get install git $ sudo apt-get install python-setuptools python-dev python-pip build-esse ...
- Solving Docker permission denied while trying to connect to the Docker daemon socket
The error message tells you that your current user can’t access the docker engine, because you’re la ...
- C# 动态语言扩展（11）
在 C# 4 开始添加 dynamic 类型.Mono C# 已经支持 C# 6.0 了. DLR C# 4 动态功能是 Dynamic Language Runtime (动态语言运行时,DLR)的 ...
- service worker在移动端H5项目的应用
1. PWA和Service Worker的关系 PWA (Progressive Web Apps) 不是一项技术,也不是一个框架,我们可以把她理解为一种模式,一种通过应用一些技术将 Web App ...
- Spring@PostConstruct和@PreDestroy注解详解
@PostConstruct注解使用 @PostConstructApi使用说明 The PostConstruct annotation is used on a method that needs ...
- 北京清北 综合强化班 Day5
T1 思路: 输入数据,sort一下, 如果a[i]>sum+1(前缀和) 那么sum+1就一定不会被拼出来, 然后输出即可. 上代码: #include <iostream> #i ...