IP SERVICES
192.168.72.55 keepalived+nginx
192.168.72.56 keepalived+nginx
192.168.72.57 keepalived+nginx
192.168.72.100 VIP
  • 如果需要给kube-apiserver提供高可用,建议在部署kubernetes集群的时候就预先部署好keepalived+nginx,

    • 否则的话,需要修改kube-apiserver的证书,将vip加入到证书内,
    • 并且修改kube-config相关的秘钥(kube-controllerkube-schedulerkubeletkube-proxy)都需要重新生成,将原有的kube-apiserver的认证地址修改为VIP
    • 重启kube-apiserverkube-controllerkube-schedulerkubeletkube-proxy服务,最终就可以实现高可用
    • 所以,在kubernetes集群部署前先完成keepalived+nginx就会方便很多
  • 为什么选择keepalived?
    • 因为keepalived有后端服务健康检测机制,检测到后端的nginx(nginx为了使用upstream模块实现负载均衡)服务有故障,就会将自己(keepalived)关闭,使VIP漂移到另外两个节点中的一个,用来保证和实现kube-apiserver服务的高可用
  • 此次部署,是基于了我博客前面的 suse 12 二进制部署 Kubernetets 1.19.7系列,最后的分发脚本和启动服务里面的数组需要自行修改(修改成自己的ip或者主机名即可,主机名需要提前做好hosts解析)

编译部署nginx

下载nginx源码包

k8s-01:~ # cd /opt/k8s/packages/
k8s-01:/opt/k8s/packages # wget http://nginx.org/download/nginx-1.16.1.tar.gz
k8s-01:/opt/k8s/packages # tar xf nginx-1.16.1.tar.gz

编译nginx

k8s-01:~ # cd /opt/k8s/packages/nginx-1.16.1/
k8s-01:/opt/k8s/packages/nginx-1.16.1 # ./configure --prefix=$(pwd)/nginx-prefix \
--with-stream \
--without-http \
--without-http_uwsgi_module && \
make && \
make install

配置nginx.conf

k8s-01:~ # cd /opt/k8s/conf/
k8s-01:/opt/k8s/conf # cat > kube-nginx.conf <<EOF
worker_processes 1;
events {
worker_connections 1024;
}
stream {
upstream backend {
hash \$remote_addr consistent;
server 192.168.72.55:6443 max_fails=3 fail_timeout=30s;
server 192.168.72.56:6443 max_fails=3 fail_timeout=30s;
server 192.168.72.57:6443 max_fails=3 fail_timeout=30s;
}
server {
listen *:8443;
proxy_connect_timeout 1s;
proxy_pass backend;
}
}
EOF

配置nginx为systemctl管理

k8s-01:~ # cd /opt/k8s/conf/
k8s-01:/opt/k8s/conf # cat > kube-nginx.service <<EOF
[Unit]
Description=kube-apiserver nginx proxy
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=forking
ExecStartPre=/opt/k8s/server/kube-nginx/sbin/nginx \
-c /opt/k8s/server/kube-nginx/conf/kube-nginx.conf \
-p /opt/k8s/server/kube-nginx -t
ExecStart=/opt/k8s/server/kube-nginx/sbin/nginx \
-c /opt/k8s/server/kube-nginx/conf/kube-nginx.conf \
-p /opt/k8s/server/kube-nginx
ExecReload=/opt/k8s/server/kube-nginx/sbin/nginx \
-c /opt/k8s/server/kube-nginx/conf/kube-nginx.conf \
-p /opt/k8s/server/kube-nginx -s reload
PrivateTmp=true
Restart=always
RestartSec=5
StartLimitInterval=0
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF

分发nginx二进制文件和配置文件

#!/usr/bin/env bash
source /opt/k8s/bin/k8s-env.sh for host in ${MASTER_IPS[@]}
do
printf "\e[1;34m${host}\e[0m\n"
ssh root@${host} "mkdir -p /opt/k8s/server/kube-nginx/{conf,logs,sbin}"
scp /opt/k8s/packages/nginx-1.16.1/nginx-prefix/sbin/nginx ${host}:/opt/k8s/server/kube-nginx/sbin/
scp /opt/k8s/conf/kube-nginx.conf ${host}:/opt/k8s/server/kube-nginx/conf/
scp /opt/k8s/conf/kube-nginx.service ${host}:/etc/systemd/system/
done

启动kube-nginx服务

#!/usr/bin/env bash
source /opt/k8s/bin/k8s-env.sh for host in ${MASTER_IPS[@]}
do
printf "\e[1;34m${host}\e[0m\n"
ssh root@${host} "systemctl daemon-reload && \
systemctl enable kube-nginx --now && \
systemctl status kube-nginx | grep Active"
done

编译部署keepalived

下载keepalived源码包

k8s-01:~ # cd /opt/k8s/packages/
k8s-01:/opt/k8s/packages # wget https://www.keepalived.org/software/keepalived-2.2.0.tar.gz
k8s-01:/opt/k8s/packages # tar xf keepalived-2.2.0.tar.gz

编译keepalived

k8s-01:/opt/k8s/packages/keepalived-2.2.0 # ./configure --prefix=$(pwd)/keepalived-prefix && \
make && \
make install

配置keepalived.conf

k8s-01:~ # cd /opt/k8s/conf/
k8s-01:/opt/k8s/conf # cat > keepalived.conf.template <<EOF
! Configuration File for keepalived
global_defs {
}
vrrp_script chk_nginx {
script "/etc/keepalived/check_port.sh 8443"
interval 2
weight -20
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 251
priority 100
advert_int 1
mcast_src_ip ##NODE_IP##
nopreempt
authentication {
auth_type PASS
auth_pass 11111111
}
track_script {
chk_nginx
}
virtual_ipaddress {
192.168.72.100
}
}
EOF
  • 为了避免keepalived服务出现问题,修复后重启keepalived,出现IP漂移回来的情况,这里选择了3个都是BACKUP的模式,减少数据的丢失

创建健康检测脚本

k8s-01:~ # cd /opt/k8s/conf/
k8s-01:/opt/k8s/conf # cat > check_port.sh <<"EOF"
CHK_PORT=$1
if [ -n "$CHK_PORT" ];then
PORT_PROCESS=$(ss -lt|grep $CHK_PORT|wc -l)
if [ $PORT_PROCESS -eq 0 ];then
echo "Port $CHK_PORT Is Not Used,End."
exit 1
fi
else
echo "Check Port Cant Be Empty!"
fi
EOF

配置keepalived为systemctl管理

k8s-01:~ # cd /opt/k8s/conf/
k8s-01:/opt/k8s/conf # cat > keepalived.service <<EOF
[Unit]
Description=LVS and VRRP High Availability Monitor
After=syslog.target network-online.target [Service]
Type=forking
PIDFile=/var/run/keepalived.pid
KillMode=process
EnvironmentFile=-/etc/sysconfig/keepalived
ExecStart=/usr/sbin/keepalived \$KEEPALIVED_OPTIONS
ExecReload=/bin/kill -HUP \$MAINPID [Install]
WantedBy=multi-user.target
EOF

分发keepalived二进制文件和配置文件

#!/usr/bin/env bash
source /opt/k8s/bin/k8s-env.sh for (( i=0; i < 3; i++ ))
do
sed -e "s/##NODE_IP##/${MASTER_IPS[i]}/" /opt/k8s/conf/keepalived.conf.template > \
/opt/k8s/conf/keepalived.conf-${MASTER_IPS[i]}.template
done for host in ${MASTER_IPS[@]}
do
printf "\e[1;34m${host}\e[0m\n"
ssh root@${host} "mkdir -p /etc/keepalived"
scp /opt/k8s/packages/keepalived-2.2.0/keepalived-prefix/sbin/keepalived ${host}:/usr/sbin/
scp /opt/k8s/packages/keepalived-2.2.0/keepalived-prefix/etc/sysconfig/keepalived ${host}:/etc/sysconfig/
scp /opt/k8s/conf/keepalived.conf-${host}.template ${host}:/etc/keepalived/keepalived.conf
scp /opt/k8s/conf/check_port.sh ${host}:/etc/keepalived/
scp /opt/k8s/conf/keepalived.service ${host}:/etc/systemd/system/
done for host in ${MASTER_IPS[@]}
do
printf "\e[1;34m${host}\e[0m\n"
ssh root@${host} "systemctl daemon-reload && \
systemctl enable keepalived --now && \
systemctl status keepalived | grep Active"
done

查看VIP所在的机器以及是否ping通

#!/usr/bin/env bash
source /opt/k8s/bin/k8s-env.sh for host in ${MASTER_IPS[@]}
do
printf "\e[1;34m${host}\e[0m\n"
ssh root@${host} "ip a | grep 192.168.72.100"
done ping 192.168.72.100 -c 1

suse 12 编译部署Keepalived + nginx 为 kube-apiserver 提供高可用的更多相关文章

  1. Keepalived+Nginx+tomcat实现系统的高可用

    Keepalived+Nginx+tomcat实现系统的高可用 1:安装vmware虚拟机 2:安装linux系统,我自己下载的centos6.5 3:安装JDK,tomcat 解压tomcat到/u ...

  2. 高可用高性能分布式文件系统FastDFS进阶keepalived+nginx对多tracker进行高可用热备

    在上一篇 分布式文件系统FastDFS如何做到高可用 中已经介绍了FastDFS的原理和怎么搭建一个简单的高可用的分布式文件系统及怎么访问. 高可用是实现了,但由于我们只设置了一个group,如果现在 ...

  3. 编译安装keepalived,实现双主mysql高可用

    安装keepalived 1.官网下载源码包,解压 # wget http://www.keepalived.org/software/keepalived-1.1.20.tar.gz # tar x ...

  4. LVS + Keepalived + Nginx基于DR模式构建高可用方案

    在大型网站中一般服务端会做集群,同时利用负载均衡器做负载均衡.这样有利于将大量的请求分散到各个服务器上,提升网站的响应速度.当然为了解决单点故障的问题,还会做热备份方案.这里演示利用LVS做负载均衡器 ...

  5. keepalived+nginx负载均衡+ApacheWeb实现高可用

    1.Keepalived高可用软件 Keepalived软件起初是专为LVS负载均衡软件设计的,用来管理并监控LVS集群系统中各个服务节点的状态,后来又加入了可以实现高可用的VRRP功能.因此,kee ...

  6. suse 12 编译部署 Nginx

    文章目录 编译前准备 创建nginx用户 下载nginx源码包 安装编译环境依赖 编译nginx 配置nginx为systemctl管理 Linux:~ # cat /etc/os-release N ...

  7. suse 12 二进制部署 Kubernetets 1.19.7 - 番外篇 - 增加node节点

    文章目录 0.前景提要 1.准备node节点环境 1.0.修改配置脚本参数 1.1.配置免密 1.2.添加hosts解析 1.3.修改主机名 1.4.更新PATH变量 1.5.安装依赖包 1.6.关闭 ...

  8. suse 12 二进制部署 Kubernetets 1.19.7 - 第01章 - 创建CA证书和kubectl集群管理命令

    文章目录 1.kubernetes集群部署 1.0.创建CA证书和秘钥 1.0.0.安装cfssl工具 1.0.1.创建根证书 1.0.2.创建证书签名请求文件 1.0.3.生成CA证书和秘钥 1.0 ...

  9. suse 12 二进制部署 Kubernetets 1.19.7 - 第06章 - 部署kube-apiserver组件

    文章目录 1.6.部署kube-apiserver 1.6.0.创建kubernetes证书和私钥 1.6.1.生成kubernetes证书和私钥 1.6.2.创建metrics-server证书和私 ...

随机推荐

  1. c#多进程通讯,今天,它来了

    引言 在c#中,可能大多数人针对于多线程之间的通讯,是熟能生巧,对于AsyncLocal 和ThreadLocal以及各个静态类中支持线程之间传递的GetData和SetData方法都是信手拈来,那多 ...

  2. POJ2891Strange Way to Express Integers

    http://poj.org/problem?id=2891 实际上就是一个一元线性同余方程组.按照合并的方式来解即可. 有一个注意点,调用函数是会慢的. #include<iostream&g ...

  3. 信不信由你!iPhone6屏幕宽度不一定是375px,iPhone6 Plus屏幕宽度不一定是414px

    看到这个题目你可能不信,引出这个问题的缘由是几次项目中Chrome模拟器和iPhone6真机预览效果不一致. 为什么在Chrome Emulation模拟手机页面和真机预览效果不一致? 以前觉得不外乎 ...

  4. Java 在PDF中添加工具提示|ToolTip

    本文,将介绍如何通过Java后端程序代码在PDF中创建工具提示.添加工具提示后,当鼠标悬停在页面上的元素时,将显示工具提示内容. 导入jar包 本次程序中使用的是 Free Spire.PDF for ...

  5. 【记录一个问题】神坑,自定义一个golang的error类型,居然运行崩溃了

    2020-05-20 18:20补充: 感谢yif同学提供指导,出现错误并且打印大量信息的原因是函数递归调用导致栈溢出. 而导致递归调用的关键代码是%v 类型实现了error的interface %v ...

  6. C# 同步 异步 回调 状态机 async await Demo

    源码 https://gitee.com/s0611163/AsyncAwaitDemo 为什么会研究这个? 我们项目的客户端和服务端通信用的是WCF,我就想,能不能用异步的方式调用WCF服务呢?或者 ...

  7. 怎么重载网卡?ip修改 HHS服务器

    目录 一:目录结构知识详述 1.网卡配置文件 2,ip修改 3.重载网卡信息 4.关闭网络管理器(因为已经有了network)所有要关闭NetworkManager不然会发生冲突 5.判断SSH服务是 ...

  8. 学习JAVAWEB第十三天

    会话技术 1. 会话:一次会话中包含多次请求和响应. * 一次会话:浏览器第一次给服务器资源发送请求,会话建立,直到有一方断开为止 2. 功能:在一次会话的范围内的多次请求间,共享数据 3. 方式: ...

  9. python 小兵 三元运算符

    1 if 条件成立: 2 val = 1 3 else: 4 val = 2 改成三元运算: val = 1 if 条件成立 else 2 举例 条件成立走左边,条件成立走右边 a = 2 b = 5 ...

  10. 业务4P分析实践

    原创不易,求分享.求一键三连 前言 假期继续思考BI未来的方向,其实常规的BI规划也不是不能出,比如公司BI看板建设.数据指标体系建设.业务线UE模型数据监控等,但这种不会超出预期的东西自然也能算技术 ...