kube-nginx 和 keepalived 部署安装
简介
本集群使用 nginx + keepalived 实现高可用
nginx 安装配置
下载编译nginx
nginx 只需要编译一次,把编译后的 文件拷贝到其他master机器上即可
cd /opt/k8s/work
wget http://nginx.org/download/nginx-1.15.3.tar.gz
tar -xzvf nginx-1.15.3.tar.gz
#编译
cd /opt/k8s/work/nginx-1.15.3
mkdir nginx-prefix
./configure --with-stream --without-http --prefix=$(pwd)/nginx-prefix --without-http_uwsgi_module
make && make install
#############
--without-http_scgi_module --without-http_fastcgi_module
--with-stream:开启 4 层透明转发(TCP Proxy)功能;
--without-xxx:关闭所有其他功能,这样生成的动态链接二进制程序依赖最小;
查看 nginx 动态链接的库:
[root@node01 nginx-1.15.3]# ldd ./nginx-prefix/sbin/nginx
linux-vdso.so.1 => (0x00007ffee18cc000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f5e89daa000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5e89b8e000)
libc.so.6 => /lib64/libc.so.6 (0x00007f5e897c0000)
/lib64/ld-linux-x86-64.so.2 (0x00007f5e89fae000)
由于只开启了 4 层透明转发功能,所以除了依赖 libc 等操作系统核心 lib 库外,没有对其它 lib 的依赖(如 libz、libssl 等),这样可以方便部署到各版本操作系统中
创建目录结构
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
ssh root@${node_ip} "mkdir -p /opt/k8s/kube-nginx/{conf,logs,sbin}"
done
拷贝二进制程序到其他主机 (有报错执行2遍就可以)
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
scp /opt/k8s/work/nginx-1.15.3/nginx-prefix/sbin/nginx root@${node_ip}:/opt/k8s/kube-nginx/sbin/kube-nginx
ssh root@${node_ip} "chmod a+x /opt/k8s/kube-nginx/sbin/*"
done
配置Nginx文件,开启4层透明转发
cd /opt/k8s/work
cat > kube-nginx.conf <<EOF
worker_processes 1;
events {
worker_connections 1024;
}
stream {
upstream backend {
hash $remote_addr consistent;
server 10.0.20.11:6443 max_fails=3 fail_timeout=30s;
server 10.0.20.12:6443 max_fails=3 fail_timeout=30s;
server 10.0.20.13:6443 max_fails=3 fail_timeout=30s;
}
server {
listen *:8443;
proxy_connect_timeout 1s;
proxy_pass backend;
}
}
EOF
#这里需要将server替换我们自己的地址
分发配置文件
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
scp kube-nginx.conf root@${node_ip}:/opt/k8s/kube-nginx/conf/kube-nginx.conf
done
配置Nginx启动文件
cd /opt/k8s/work
cat > kube-nginx.service <<EOF
[Unit]
Description=kube-apiserver nginx proxy
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=forking
ExecStartPre=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx -t
ExecStart=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx
ExecReload=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx -s reload
PrivateTmp=true
Restart=always
RestartSec=5
StartLimitInterval=0
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
分发nginx启动文件
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
scp kube-nginx.service root@${node_ip}:/etc/systemd/system/
done
启动 kube-nginx 服务
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
ssh root@${node_ip} "systemctl daemon-reload && systemctl enable kube-nginx && systemctl start kube-nginx"
done
检查 kube-nginx 服务运行状态
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
ssh root@${node_ip} "systemctl status kube-nginx |grep 'Active:'"
done
[root@node01 work]# for node_ip in ${MASTER_IPS[@]}
> do
> echo ">>> ${node_ip}"
> ssh root@${node_ip} "systemctl status kube-nginx |grep 'Active:'"
> done
>>> 10.0.20.11
Active: active (running) since Thu 2019-12-05 15:13:19 CST; 3s ago
>>> 10.0.20.12
Active: active (running) since Thu 2019-12-05 15:13:19 CST; 3s ago
>>> 10.0.20.13
Active: active (running) since Thu 2019-12-05 15:13:19 CST; 3s ago
检查 kube-nginx 端口
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
ssh root@${node_ip} "netstat -lntup | grep 8443"
done
[root@node01 work]# for node_ip in ${MASTER_IPS[@]}
> do
> echo ">>> ${node_ip}"
> ssh root@${node_ip} "netstat -lntup | grep 8443"
> done
>>> 10.0.20.11
tcp 0 0 0.0.0.0:8443 0.0.0.0:* LISTEN 5356/nginx: master
>>> 10.0.20.12
tcp 0 0 0.0.0.0:8443 0.0.0.0:* LISTEN 2586/nginx: master
>>> 10.0.20.13
tcp 0 0 0.0.0.0:8443 0.0.0.0:* LISTEN 2630/nginx: master
keepalived 安装配置
安装keeplive服务
前面我们也说了,高可用方案需要一个VIP,供集群内部访问
在所有master节点安装keeplived
yum install -y keepalived
配置keeplive服务
配置文件模板
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
cat > keepalived.conf.template <<EOF
! Configuration File for keepalived
global_defs {
router_id ##MASTER_IP##
}
vrrp_script chk_nginx {
script "/etc/keepalived/check_port.sh 8443"
interval 2
weight -20
}
vrrp_instance VI_1 {
state MASTER
interface ##IFACE##
virtual_router_id 251
priority 100
advert_int 1
mcast_src_ip ##MASTER_IP##
nopreempt
authentication {
auth_type PASS
auth_pass 11111111
}
track_script {
chk_nginx
}
virtual_ipaddress {
##KEEP_VIP##
}
}
EOF
替换模板文件的变量,为各个节点生成配置文件
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for (( i=0; i < 3; i++ ))
do
sed -e "s/##MASTER_IP##/${MASTER_IPS[i]}/" -e "s/##KEEP_VIP##/${KEEP_VIP_ADDR}/" -e "s/##IFACE##/${IFACE}/" keepalived.conf.template > keepalived-${MASTER_IPS[i]}.conf
done
ls keepalived-*.conf
将对应的keepalived配置文件拷贝到对应的节点上
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for (( i=0; i < 3; i++ ))
do
echo ">>> ${node_ip}"
scp keepalived-${MASTER_IPS[i]}.conf ${MASTER_NAMES[i]}:/etc/keepalived/keepalived.conf
done
创建健康检查脚本
cd /opt/k8s/work
cat > check_port.sh <<EOF
#!/bin/sh
CHK_PORT=\$1
if [ -n "\$CHK_PORT" ];then
PORT_PROCESS=\`ss -lntup|grep \${CHK_PORT}|wc -l\`
if [ \$PORT_PROCESS -eq 0 ];then
echo -e "\033[31m ERROR: Port \$CHK_PORT Is Not Used,End. \033[0m"
exit 1
fi
fi
EOF
分发脚本到所有keepalived节点
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for node in ${MASTER_IPS[@]}
do
echo ">>> ${node}"
scp check_port.sh ${node}:/etc/keepalived/
done
启动keeplived
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for node in ${MASTER_IPS[@]}
do
echo ">>> ${node}"
ssh ${node} "systemctl enable keepalived && systemctl start keepalived && systemctl status keepalived | grep active"
done
查看VIP地址
cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
for node in ${MASTER_IPS[@]}
do
echo ">>> ${node}"
ssh ${node} "ip a | grep 20.10"
done
输出结果
[root@node01 work]# for node in ${MASTER_IPS[@]}
> do
> echo ">>> ${node}"
> ssh ${node} "ip a | grep 20.10"
> done
>>> 10.0.20.11
>>> 10.0.20.12
>>> 10.0.20.13
inet 10.0.20.10/32 scope global bond0
kube-nginx 和 keepalived 部署安装的更多相关文章
- docker 部署nginx 使用keepalived 部署高可用
一.体系架构 在Keepalived + Nginx高可用负载均衡架构中,keepalived负责实现High-availability (HA) 功能控制前端机VIP(虚拟网络地址),当有设备发生故 ...
- FastDFS+Nginx分布式文件系统部署安装
安装: ##安装tracker.storage参考:http://www.cnblogs.com/zclzhao/p/5025229.html nginx代理设置参考:http://www.cnblo ...
- Tomcat+nginx+Keepalived部署实现集群
Tomcat+nginx+Keepalived部署实现集群 环境说明: 系统:Centos-7 主机:Centos-7 x3 IP地址: 服务器1(192.168.10.102/24) 服务器2(19 ...
- Nginx 集群部署(Keepalived)
# Nginx集群部署 # 当我们的用户同时访问量达到一定量的时候,一台服务器是不够用的 # 这个时候我们需要解决这个问题肯定是要添加新的服务器去处理用户访问 # 多台服务器处理用户访问就需要我们集群 ...
- Linux、Debian、Jenkins、GIT、Nginx、码云安装,自动化部署前后端分离项目
1.安装Jenkins i:下载Jenkins安装包(war文件):https://jenkins.io/download/ ii:这里采用Tomcat的war包方式安装,讲下载好的安装包放到Tomc ...
- Nginx+keepalived 脚本安装主从双机热备自动切换解决方案
Nginx+keepalived 脚本安装主从双机热备自动切换解决方案 2013-07-02 19:24:13 标签:filesnginx keepalived 原创作品,允许转载,转载时请务必以超链 ...
- nginx部署安装
首先需要下载Nginx软件包 nginx软件官方下载地址:[nginx官方下载连接](http://www.nginx.org) 建议选择稳定的软件版本,如果练习使用当然是无所谓,随便什么版本都可以, ...
- linux安装Nginx 以及 keepalived 管理Nginx
linux安装Nginx 1.1将Nginx素材内容上传到/usr/local目录(pcre,zlib,openssl,nginx)(注意:必须登录用对这个文件具有操作权限的) 1.2安装pcre库 ...
- 阿里云安装Nginx+vue项目部署
阿里云安装Nginx+vue项目部署 nginx安装包下载 http://nginx.org/en/download.html nginx安装 首先先安装PCRE pcre-devel 和Zlib,因 ...
随机推荐
- "Dependency on app with no migrations: %s" % key[0]
问题描述:我在model中建好模型类,运行的控制台就报错误: 解决方法:1,首先需要在setting中重载AUTH_USER_MODEL AUTH_USER_MODEL = 'users.UserPr ...
- .net core 3.0 在过滤器读取request.body 里的请求,controller[FromBody]读取不到参数,解决办法
1,注入IHttpContextAccessor httpContex 2,var req = _httpContext.HttpContext.Request; // 这句很重要,开启读取 否者下 ...
- 一款用于绘制状态机转换图和流程图的web在线绘图工具
大型软件系统中离不开各类状态机的处理,日常工作中也涉及到各类事务处理流程:从表现力看文不如表,表不如图:因此日常工作中经常需要绘制各种状态机的状态转换图和流程图,以协助理解代码逻辑和各类事务处理流程等 ...
- find 小案例
说明:前几天对生产环境的一些重要数据进行备份时用到了find,查找特定符合条件的文件名后拷贝至指定目录,但是只拷贝了部分匹配到的文件 小案例模拟还原: [root@centos- ~]# ll /te ...
- Java 干货之深入理解Java内部类
可以将一个类定义在另一个类或方法中,这样的类叫做内部类 --<Thinking in Java> 说起内部类,大家并不陌生,并且会经常在实例化容器的时候使用到它.但是内部类的具体细节语法, ...
- C/C.cpp
我是怎么说服自己这题没有答案单调性的? 当轮数大于最优答案时,B的操作就成了捣乱,会使A多进行若干操作才能回复原状态 嗯,所以答案没有单调性. 就没去想A的操作可以把B的抵消啊.. 另外swap没有交 ...
- 安装cnpm遇到的问题
安装 cnpm时,用git安装时,安装好node环境后,测试版本号node -v和npm -v都没问题,可以输出版本号,但是安装cnpm时,使用淘宝镜像安装后,会出现如下警告: 这个是提醒你安装的版本 ...
- 「Luogu 1525」关押罪犯
更好的阅读体验 Portal Portal1: Luogu Portal2: LibreOJ Description \(S\)城现有两座监狱,一共关押着\(N\)名罪犯,编号分别为\(1 - N\) ...
- C++程序员学Python
目录 C++程序员学Python 第二章.变量和数据类型 1.注释语句前用#: 2.常用于大小写函数: 第三章.列表 1.列表简述 2.修改,增加,插入,删除列表元素 第四章操作列表 1.遍历 2.创 ...
- UEFI+GPT电脑Win10下安装openSUSE Leap 42.2双系统
安装过程仅供参考,最后实现方式不完美. 1 准备工具,一个8G以上U盘,已装好win10的UEFI+GPT电脑(本机为SSD+HDD双硬盘) 2 所需软件: 2.1 ...