实战项目部署应用到kubernetes流程(jenkins+docker+k8s)
说明
通过jenkins构建java应用程序发布到k8s集群中
本文已一个大数据的java项目来演示构建部署过程
支持发布和回滚
支持一套模板应用不同项目
k8s基础准备
创建项目名称空间
[root@k8s-master ~]# kubectl create ns architect
namespace/architect created [root@k8s-master ~]# kubectl label namespace architect istio-injection=enabled
namespace/architect labeled
[root@k8s-master ~]# kubectl get namespace -L istio-injection
NAME STATUS AGE ISTIO-INJECTION
architect Active 87s enabled
default Active 39d enabled
istio-system Active 7d17h
kube-public Active 39d
kube-system Active 39d
[root@k8s-master ~]#
创建全局配置文件
＃创建大数据配置文件
[root@k8s-master ~]# ls bigdata-counter
core-site.xml dmp_operator1.keytab hbase-site.xml hdfs-site.xml krb5.conf
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl create configmap bigdata-config --from-file=/root/bigdata-counter/ -n architect
configmap/bigdata-config created
[root@k8s-master ~]#
#全局环境变量
[root@k8s-master ~]# cat yaml/global-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: global-config
namespace: architect
data:
CONFIG_SERVER_URL: http://config.ms.xx.com
CONFIG_SERVER_USER: myconfig
ILOG_BOOTSTRAP_SERVERS: bootstrap.servers=192.168.1.122:9092
CONFIG_SERVER_PWDS: mc123456 [root@k8s-master ~]#
[root@k8s-master ~]# kubectl apply -f yaml/global-config.yaml
configmap/global-config created
[root@k8s-master ~]# kubectl get configmap -n architect
NAME DATA AGE
bigdata-config 4 2m33s
global-config 4 3s
[root@k8s-master ~]#
创建habror认证信息
[root@k8s-master ~]# kubectl create secret docker-registry registry-secret --docker-server=dev-hub.xx.net --docker-username=admin --docker-password=Harbor12345 --docker-email=admin@dev-hub.xx.net -n architect
secret/registry-secret created
[root@k8s-master ~]# kubectl get secret -n architect
NAME TYPE DATA AGE
default-token-5wsxl kubernetes.io/service-account-token 3 2m24s
istio.default istio.io/key-and-cert 3 2m24s
registry-secret kubernetes.io/dockerconfigjson 1 10s
项目deployment文件
可以做成模本，通过变量替换形式．后面有讲到
[root@k8s-master ~]# cat bigdata-interface-deploy.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: bigdata-interface
version: default
name: bigdata-interface
namespace: architect
spec:
replicas: 1
selector:
matchLabels:
app: bigdata-interface
version: default
template:
metadata:
labels:
app: bigdata-interface
version: default
spec:
containers:
- env:
- name: izone
- name: MY_SERVICE_NAME
value: bigdata-interface
- name: ilogEnvs
value: izone,buildNumber
- name: ILOG_BOOTSTRAP_SERVERS
valueFrom:
configMapKeyRef:
key: ILOG_BOOTSTRAP_SERVERS
name: global-config
- name: CONFIG_SERVER_URL
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_URL
name: global-config
- name: CONFIG_SERVER_USER
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_USER
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: JAR_BOOT_ARG
value: --spring.cloud.config.fail-fast=true
- name: JDK_HEAP_OPTS
value: -Xms256m -Xmx256m
image: dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b63
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 2
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 300
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: bigdata-interface
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
volumeMounts:
- mountPath: /opt/jdk
name: jdk-path
- mountPath: /srv/applogs
name: applogs
imagePullSecrets:
- name: registry-secret
restartPolicy: Always
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /usr/local/jdk/default
type: ""
name: jdk-path
- hostPath:
path: /data/applogs
type: ""
name: applogs
[root@k8s-master ~]#
jenkins pipeline 模板文件
node {
//项目名称
String APPNAME='bigdata-interface'
//名称空间
String NAMESPACE='architect'
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(name:'GIT_BRANCH_TAG', defaultValue: 'master',description:'', trim: false)
])])
stage('git clone') {
checkout([$class: 'GitSCM', branches: [[name: params.GIT_BRANCH_TAG]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bc', url: 'git@git.jiatuiyun.net:bigdata-center/bigdata-interface.git']]])
}
stage('maven build') {
sh 'mvn clean package -U'
}
stage('docker build && push habror'){
sh "cd ${WORKSPACE}"
restapi = '`ls -d *-restapi||echo .`'
sh "ls -l ${restapi}/target/"
sh "echo ${restapi}"
sh "curl -o ${restapi}/target/Dockerfile http://192.168.1.55:8089/files/Dockerfile-java.v1"
sh "docker build -t ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${restapi}/target"
sh "docker tag ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
sh "docker login ${HarborUrl} -u ${HarborUser} -p ${HarborPasswd}"
sh "docker push ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
}
stage('deploy'){
echo "${params.GIT_BRANCH_TAG}"
echo "${BUILD_NUMBER}"
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
进一步改造pipeline
node {
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: 'git@git.jiatuiyun.net:bigdata-center/bigdata-interface.git', description: 'git仓库地址', name: 'XBUILD_REPO', trim: false),
string(defaultValue: 'master', description: '版本名称', name: 'XBUILD_BRANCHE', trim: false)
])])
stage('git clone') {
checkout([$class: 'GitSCM', branches: [[name: params.XBUILD_BRANCHE]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bc', url: params.XBUILD_REPO]]])
}
stage('maven build') {
sh 'mvn clean package -U'
}
stage('docker build && push habror'){
sh "cd ${WORKSPACE}"
restapi = '`ls -d *-restapi||echo .`'
sh "ls -l ${restapi}/target/"
sh "echo ${restapi}"
sh "curl -o ${restapi}/target/Dockerfile http://192.168.1.55:8089/files/Dockerfile-java.v1"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
sh "docker build -t ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${restapi}/target"
sh "docker tag ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
sh "docker login ${HarborUrl} -u ${HarborUser} -p ${HarborPasswd}"
sh "docker push ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
}
stage('deploy'){
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
echo "${params.GIT_BRANCH_TAG}"
echo "${BUILD_NUMBER}"
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
//updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
jenkins上创建自由风格项目
效果如下 支持参数化构建。但每项目单独JOB
查看镜像仓库已经推送正常
项目回滚
node {
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: '1', description: '镜像标签', name: 'XBUILD_NUMBER', trim: false)
])])
stage('deploy'){
echo "${BUILD_NUMBER}"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
BUILD_NUMBER = params.XBUILD_NUMBER
echo "----${params.XBUILD_NUMBER}---${params.XBUILD_DEPLOY} ----${params.XBUILD_TEAM} "
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
//updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
jenkins效果如下：输入namespace和deployment 名称和上次构建版本
查看k8s集群
可以灵活的在已知版本回滚操作
[root@k8s-master ~]# kubectl get deploy -n architect -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
bigdata-interface 0/1 1 0 3h12m bigdata-interface dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b1 app=bigdata-interface,version=default
[root@k8s-master ~]# kubectl get pods -n architect
NAME READY STATUS RESTARTS AGE
bigdata-interface-6bf7b5646d-sf9j5 1/2 Running 0 24s
[root@k8s-master ~]#
通过以上实践，可以将deployment做成模板
deployment模板文件
-bash-4.2$ cat k8s-deployment.tpl
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: {APPNAME}
version: default
name: {APPNAME}
namespace: {NAMESPACE}
spec:
replicas: 1
selector:
matchLabels:
app: {APPNAME}
version: default
template:
metadata:
labels:
app: {APPNAME}
version: default
spec:
containers:
- env:
- name: izone
- name: MY_SERVICE_NAME
value: {APPNAME}
- name: ilogEnvs
value: izone,buildNumber
- name: ILOG_BOOTSTRAP_SERVERS
valueFrom:
configMapKeyRef:
key: ILOG_BOOTSTRAP_SERVERS
name: global-config
- name: CONFIG_SERVER_URL
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_URL
name: global-config
- name: CONFIG_SERVER_USER
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_USER
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: JAR_BOOT_ARG
value: --spring.cloud.config.fail-fast=true
- name: JDK_HEAP_OPTS
value: -Xms256m -Xmx256m
image: {HarborUrl}/{NAMESPACE}/{APPNAME}:{NAMESPACE}_b{BUILD_NUMBER}
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 2
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 300
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: {APPNAME}
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
volumeMounts:
- mountPath: /opt/jdk
name: jdk-path
- mountPath: /srv/applogs
name: applogs
imagePullSecrets:
- name: registry-secret
restartPolicy: Always
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /usr/local/jdk/default
type: ""
name: jdk-path
- hostPath:
path: /data/applogs
type: ""
name: applogs
-bash-4.2$
通过模板动态生成项目的deploy文件．进行升级回滚．
比如下面的：
node {
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: '1', description: '镜像标签', name: 'XBUILD_NUMBER', trim: false)
])])
stage('deploy'){
echo "${BUILD_NUMBER}"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
BUILD_NUMBER = params.XBUILD_NUMBER
echo "----${params.XBUILD_NUMBER}---${params.XBUILD_DEPLOY} ----${params.XBUILD_TEAM} "
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
command="sed -e 's#{APPNAME}#${APPNAME}#g;s#{NAMESPACE}#${NAMESPACE}#g;s#{HarborUrl}#${HarborUrl}#g;s#{BUILD_NUMBER}#${BUILD_NUMBER}#g' /data/jenkins_slave/k8s-deployment.tpl > /data/jenkins_slave/${APPNAME}-deploy.yaml"
sh "${command}"
sh "scp -P 22 /data/jenkins_slave/${APPNAME}-deploy.yaml ${KubernetHost}:/root/ "
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
sh "ssh -p 22 ${KubernetHost} kubectl apply -f /root/${APPNAME}-deploy.yaml"
//test
}
}
pipeline参数化构建
示例：
pipeline {
agent any
parameters {
gitParameter branch: '', branchFilter: '.*', defaultValue: '', description: '', name: 'XBUILD_BRANCHE', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH_TAG'
string defaultValue: 'git@git.xx.net:company/xx-xx.git', description: '', name: 'XBUILD_REPO', trim: false
}
stages {
stage('Example') {
steps {
checkout([$class: 'GitSCM', branches: [[name: params.XBUILD_BRANCHE]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bb', url: params.XBUILD_REPO]]])
}
}
}
}
构建效果如下：
实战项目部署应用到kubernetes流程(jenkins+docker+k8s)的更多相关文章
- docker 实战-项目部署
前面学习了 docker 的命令和实际的用法,现在来实战一下.编排一个服务,也就是项目部署. 目前我们在一个闭源环境下工作,介绍一些工作流程和部署项目的方法. 该工作流程比较特殊 所谓闭源环境就是 没 ...
- jenkins+docker+k8s项目发布
目录 一.简介 二.新建docker-build工程 三.项目部署 四.访问测试 一.简介 1.该章节基于jenkins.Harbor.pipeline.k8s来做发布,如对这些不熟悉,请按以下进入学 ...
- .Net 6 WebApi 项目部署到 Linux 系统上的 Docker 容器
1.创建一个基础的WebApi项目 注意:因为发布时候,Dockerfile文件必须和解决方案.cspro文件放在同级,所以建议勾上这个,当时遇到这个问题,导致打包镜像时找不到.cspro文件,搞了好 ...
- 第四十四章 微服务CICD（6）- gitlab + jenkins + docker + k8s
总体流程: 在开发机开发代码后提交到gitlab 之后通过webhook插件触发jenkins进行构建,jenkins将代码打成docker镜像,push到docker-registry 之后将在k8 ...
- gitlab + jenkins + docker + k8s
总体流程: 在开发机开发代码后提交到gitlab 之后通过webhook插件触发jenkins进行构建,jenkins将代码打成docker镜像,push到docker-registry 之后将在k8 ...
- kubernetes下jenkins实战maven项目编译构建
关于kubernetes环境的jenkins集群 在kubernetes环境部署的jenkins集群,执行任务时会新建pod,任务完成后pod被销毁,架构如下所示: 在kubernetes搭建jenk ...
- Kubernetes笔记（三）：Gitlab+Jenkins Pipeline+Docker+k8s+Helm自动化部署实践（干货分享！）
通过前面两篇文章,我们已经有了一个"嗷嗷待哺"的K8s集群环境,也对相关的概念与组件有了一个基本了解(前期对概念有个印象即可,因为只有实践了才能对其有深入理解,所谓"纸上 ...
- 【k8s实战一】Jenkins 部署应用到 Kubernetes
[k8s实战一]Jenkins 部署应用到 Kubernetes 01 本文主旨 目标是演示整个Jenkins从源码构建镜像到部署镜像到Kubernetes集群过程. 为了简化流程与容易重现文中效果, ...
- kubernetes部署jenkins(Docker in Docker)及认证
引言 Jenkins是一款开源 CI&CD 软件,用于自动化各种任务,包括构建.测试和部署软件. 本文将Jenkins的master与slave置于Pod中,部署在namespace:jenk ...
随机推荐
- 第31天学习打卡（File类。字符流读写文件）
File类 概念 文件,文件夹,一个file对象代表磁盘上的某个文件或者文件夹 构造方法 File(String pathname) File(String parent,String child) ...
- molloc堆区的动态内存分配
[前言]前面有一篇文章介绍了堆区栈区的区别.栈区的核心主要集中在操作一个栈结构,一般由操作系统维护.堆区,主要是我们程序员来维护,核心就是动态内存分配. 这篇笔记结束就不在高新CSAPP的读书笔记了, ...
- Hexo一键部署到阿里云OSS并设置浏览器缓存
自建博客地址:https://bytelife.net,欢迎访问! 本文为博客自动同步文章,为了更好的阅读体验,建议您移步至我的博客 本文作者: Jeffrey 本文链接: https://bytel ...
- MySQL：字段约束与索引
字段约束 MySQL的字段约束共四种: 约束名 关键字 描述 主键约束 PRIMARY KEY 字段值唯一,且不能为NULL 非空约束 NOT NULL 字段值不能为NULL 唯一约束 UNIQUE ...
- OpenGL中的简单坐标系初看+VAO/VBO/EBO
你好,三角形 一: 关于坐标的问题 标准化设备坐标:输入的顶点数据就应该在标准化设备坐标范围里面即:x,y,z的值都在(-1-1)之间.在这个区间之外的坐标都会被丢弃. 1.1一旦顶点数据传入顶点着色 ...
- POJ-1182(经典带权并查集)
食物链 POJ-1182 一个很好的分析博客:https://blog.csdn.net/niushuai666/article/details/6981689 三种关系:两者同类,吃父节点,被父节点 ...
- PTA1071 - Speech Patterns - map计算不同单词个数
题意 输出给定字符串出现最多的字符串(小写输出)和出现次数. 所求字符串要求:字符中可以含有A-Z.0-9. 比如说题目给出的Can1,我们可以转换成can1,can1就算一个字符串整体,而不是单独的 ...
- 为什么要从 Linux 迁移到 BSD2
OpenZFS on Linux,是项目的 Linux 部分,目前有 345 个活跃的贡献者,有超过 5600 个提交,而且几乎每天都有提交!一些世界上最大的 CDN 和数据存储服务在 FreeBSD ...
- Apache配置 11. 访问控制-user_agent
(1)介绍 user_agent是指用户浏览器端的信息.比如你是用IE的还是Firefox浏览器的.有些网站会根据这个来调整打开网站的类型,如是手机的就打开wap,显示非手机的就打开PC常规页面. ( ...
- 【linux】Linux删除文件后磁盘依旧占用空间的问题
转自https://blog.51cto.com/2483526/798379 感谢 1.用df 检查发现/根目录可用空间为0 [root@/]#df -h 2.用du检查发现各目录占用的空间都很少, ...