实战项目部署应用到kubernetes流程(jenkins+docker+k8s)
说明
通过jenkins构建java应用程序发布到k8s集群中
本文已一个大数据的java项目来演示构建部署过程
支持发布和回滚
支持一套模板应用不同项目
k8s基础准备
创建项目名称空间
[root@k8s-master ~]# kubectl create ns architect
namespace/architect created [root@k8s-master ~]# kubectl label namespace architect istio-injection=enabled
namespace/architect labeled
[root@k8s-master ~]# kubectl get namespace -L istio-injection
NAME STATUS AGE ISTIO-INJECTION
architect Active 87s enabled
default Active 39d enabled
istio-system Active 7d17h
kube-public Active 39d
kube-system Active 39d
[root@k8s-master ~]#
创建全局配置文件
＃创建大数据配置文件
[root@k8s-master ~]# ls bigdata-counter
core-site.xml dmp_operator1.keytab hbase-site.xml hdfs-site.xml krb5.conf
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl create configmap bigdata-config --from-file=/root/bigdata-counter/ -n architect
configmap/bigdata-config created
[root@k8s-master ~]#
#全局环境变量
[root@k8s-master ~]# cat yaml/global-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: global-config
namespace: architect
data:
CONFIG_SERVER_URL: http://config.ms.xx.com
CONFIG_SERVER_USER: myconfig
ILOG_BOOTSTRAP_SERVERS: bootstrap.servers=192.168.1.122:9092
CONFIG_SERVER_PWDS: mc123456 [root@k8s-master ~]#
[root@k8s-master ~]# kubectl apply -f yaml/global-config.yaml
configmap/global-config created
[root@k8s-master ~]# kubectl get configmap -n architect
NAME DATA AGE
bigdata-config 4 2m33s
global-config 4 3s
[root@k8s-master ~]#
创建habror认证信息
[root@k8s-master ~]# kubectl create secret docker-registry registry-secret --docker-server=dev-hub.xx.net --docker-username=admin --docker-password=Harbor12345 --docker-email=admin@dev-hub.xx.net -n architect
secret/registry-secret created
[root@k8s-master ~]# kubectl get secret -n architect
NAME TYPE DATA AGE
default-token-5wsxl kubernetes.io/service-account-token 3 2m24s
istio.default istio.io/key-and-cert 3 2m24s
registry-secret kubernetes.io/dockerconfigjson 1 10s
项目deployment文件
可以做成模本，通过变量替换形式．后面有讲到
[root@k8s-master ~]# cat bigdata-interface-deploy.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: bigdata-interface
version: default
name: bigdata-interface
namespace: architect
spec:
replicas: 1
selector:
matchLabels:
app: bigdata-interface
version: default
template:
metadata:
labels:
app: bigdata-interface
version: default
spec:
containers:
- env:
- name: izone
- name: MY_SERVICE_NAME
value: bigdata-interface
- name: ilogEnvs
value: izone,buildNumber
- name: ILOG_BOOTSTRAP_SERVERS
valueFrom:
configMapKeyRef:
key: ILOG_BOOTSTRAP_SERVERS
name: global-config
- name: CONFIG_SERVER_URL
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_URL
name: global-config
- name: CONFIG_SERVER_USER
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_USER
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: JAR_BOOT_ARG
value: --spring.cloud.config.fail-fast=true
- name: JDK_HEAP_OPTS
value: -Xms256m -Xmx256m
image: dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b63
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 2
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 300
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: bigdata-interface
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
volumeMounts:
- mountPath: /opt/jdk
name: jdk-path
- mountPath: /srv/applogs
name: applogs
imagePullSecrets:
- name: registry-secret
restartPolicy: Always
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /usr/local/jdk/default
type: ""
name: jdk-path
- hostPath:
path: /data/applogs
type: ""
name: applogs
[root@k8s-master ~]#
jenkins pipeline 模板文件
node {
//项目名称
String APPNAME='bigdata-interface'
//名称空间
String NAMESPACE='architect'
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(name:'GIT_BRANCH_TAG', defaultValue: 'master',description:'', trim: false)
])])
stage('git clone') {
checkout([$class: 'GitSCM', branches: [[name: params.GIT_BRANCH_TAG]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bc', url: 'git@git.jiatuiyun.net:bigdata-center/bigdata-interface.git']]])
}
stage('maven build') {
sh 'mvn clean package -U'
}
stage('docker build && push habror'){
sh "cd ${WORKSPACE}"
restapi = '`ls -d *-restapi||echo .`'
sh "ls -l ${restapi}/target/"
sh "echo ${restapi}"
sh "curl -o ${restapi}/target/Dockerfile http://192.168.1.55:8089/files/Dockerfile-java.v1"
sh "docker build -t ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${restapi}/target"
sh "docker tag ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
sh "docker login ${HarborUrl} -u ${HarborUser} -p ${HarborPasswd}"
sh "docker push ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
}
stage('deploy'){
echo "${params.GIT_BRANCH_TAG}"
echo "${BUILD_NUMBER}"
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
进一步改造pipeline
node {
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: 'git@git.jiatuiyun.net:bigdata-center/bigdata-interface.git', description: 'git仓库地址', name: 'XBUILD_REPO', trim: false),
string(defaultValue: 'master', description: '版本名称', name: 'XBUILD_BRANCHE', trim: false)
])])
stage('git clone') {
checkout([$class: 'GitSCM', branches: [[name: params.XBUILD_BRANCHE]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bc', url: params.XBUILD_REPO]]])
}
stage('maven build') {
sh 'mvn clean package -U'
}
stage('docker build && push habror'){
sh "cd ${WORKSPACE}"
restapi = '`ls -d *-restapi||echo .`'
sh "ls -l ${restapi}/target/"
sh "echo ${restapi}"
sh "curl -o ${restapi}/target/Dockerfile http://192.168.1.55:8089/files/Dockerfile-java.v1"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
sh "docker build -t ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${restapi}/target"
sh "docker tag ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
sh "docker login ${HarborUrl} -u ${HarborUser} -p ${HarborPasswd}"
sh "docker push ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
}
stage('deploy'){
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
echo "${params.GIT_BRANCH_TAG}"
echo "${BUILD_NUMBER}"
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
//updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
jenkins上创建自由风格项目
效果如下 支持参数化构建。但每项目单独JOB
查看镜像仓库已经推送正常
项目回滚
node {
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: '1', description: '镜像标签', name: 'XBUILD_NUMBER', trim: false)
])])
stage('deploy'){
echo "${BUILD_NUMBER}"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
BUILD_NUMBER = params.XBUILD_NUMBER
echo "----${params.XBUILD_NUMBER}---${params.XBUILD_DEPLOY} ----${params.XBUILD_TEAM} "
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
//updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
jenkins效果如下：输入namespace和deployment 名称和上次构建版本
查看k8s集群
可以灵活的在已知版本回滚操作
[root@k8s-master ~]# kubectl get deploy -n architect -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
bigdata-interface 0/1 1 0 3h12m bigdata-interface dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b1 app=bigdata-interface,version=default
[root@k8s-master ~]# kubectl get pods -n architect
NAME READY STATUS RESTARTS AGE
bigdata-interface-6bf7b5646d-sf9j5 1/2 Running 0 24s
[root@k8s-master ~]#
通过以上实践，可以将deployment做成模板
deployment模板文件
-bash-4.2$ cat k8s-deployment.tpl
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: {APPNAME}
version: default
name: {APPNAME}
namespace: {NAMESPACE}
spec:
replicas: 1
selector:
matchLabels:
app: {APPNAME}
version: default
template:
metadata:
labels:
app: {APPNAME}
version: default
spec:
containers:
- env:
- name: izone
- name: MY_SERVICE_NAME
value: {APPNAME}
- name: ilogEnvs
value: izone,buildNumber
- name: ILOG_BOOTSTRAP_SERVERS
valueFrom:
configMapKeyRef:
key: ILOG_BOOTSTRAP_SERVERS
name: global-config
- name: CONFIG_SERVER_URL
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_URL
name: global-config
- name: CONFIG_SERVER_USER
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_USER
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: JAR_BOOT_ARG
value: --spring.cloud.config.fail-fast=true
- name: JDK_HEAP_OPTS
value: -Xms256m -Xmx256m
image: {HarborUrl}/{NAMESPACE}/{APPNAME}:{NAMESPACE}_b{BUILD_NUMBER}
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 2
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 300
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: {APPNAME}
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
volumeMounts:
- mountPath: /opt/jdk
name: jdk-path
- mountPath: /srv/applogs
name: applogs
imagePullSecrets:
- name: registry-secret
restartPolicy: Always
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /usr/local/jdk/default
type: ""
name: jdk-path
- hostPath:
path: /data/applogs
type: ""
name: applogs
-bash-4.2$
通过模板动态生成项目的deploy文件．进行升级回滚．
比如下面的：
node {
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: '1', description: '镜像标签', name: 'XBUILD_NUMBER', trim: false)
])])
stage('deploy'){
echo "${BUILD_NUMBER}"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
BUILD_NUMBER = params.XBUILD_NUMBER
echo "----${params.XBUILD_NUMBER}---${params.XBUILD_DEPLOY} ----${params.XBUILD_TEAM} "
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
command="sed -e 's#{APPNAME}#${APPNAME}#g;s#{NAMESPACE}#${NAMESPACE}#g;s#{HarborUrl}#${HarborUrl}#g;s#{BUILD_NUMBER}#${BUILD_NUMBER}#g' /data/jenkins_slave/k8s-deployment.tpl > /data/jenkins_slave/${APPNAME}-deploy.yaml"
sh "${command}"
sh "scp -P 22 /data/jenkins_slave/${APPNAME}-deploy.yaml ${KubernetHost}:/root/ "
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
sh "ssh -p 22 ${KubernetHost} kubectl apply -f /root/${APPNAME}-deploy.yaml"
//test
}
}
pipeline参数化构建
示例：
pipeline {
agent any
parameters {
gitParameter branch: '', branchFilter: '.*', defaultValue: '', description: '', name: 'XBUILD_BRANCHE', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH_TAG'
string defaultValue: 'git@git.xx.net:company/xx-xx.git', description: '', name: 'XBUILD_REPO', trim: false
}
stages {
stage('Example') {
steps {
checkout([$class: 'GitSCM', branches: [[name: params.XBUILD_BRANCHE]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bb', url: params.XBUILD_REPO]]])
}
}
}
}
构建效果如下：
实战项目部署应用到kubernetes流程(jenkins+docker+k8s)的更多相关文章
- docker 实战-项目部署
前面学习了 docker 的命令和实际的用法,现在来实战一下.编排一个服务,也就是项目部署. 目前我们在一个闭源环境下工作,介绍一些工作流程和部署项目的方法. 该工作流程比较特殊 所谓闭源环境就是 没 ...
- jenkins+docker+k8s项目发布
目录 一.简介 二.新建docker-build工程 三.项目部署 四.访问测试 一.简介 1.该章节基于jenkins.Harbor.pipeline.k8s来做发布,如对这些不熟悉,请按以下进入学 ...
- .Net 6 WebApi 项目部署到 Linux 系统上的 Docker 容器
1.创建一个基础的WebApi项目 注意:因为发布时候,Dockerfile文件必须和解决方案.cspro文件放在同级,所以建议勾上这个,当时遇到这个问题,导致打包镜像时找不到.cspro文件,搞了好 ...
- 第四十四章 微服务CICD（6）- gitlab + jenkins + docker + k8s
总体流程: 在开发机开发代码后提交到gitlab 之后通过webhook插件触发jenkins进行构建,jenkins将代码打成docker镜像,push到docker-registry 之后将在k8 ...
- gitlab + jenkins + docker + k8s
总体流程: 在开发机开发代码后提交到gitlab 之后通过webhook插件触发jenkins进行构建,jenkins将代码打成docker镜像,push到docker-registry 之后将在k8 ...
- kubernetes下jenkins实战maven项目编译构建
关于kubernetes环境的jenkins集群 在kubernetes环境部署的jenkins集群,执行任务时会新建pod,任务完成后pod被销毁,架构如下所示: 在kubernetes搭建jenk ...
- Kubernetes笔记（三）：Gitlab+Jenkins Pipeline+Docker+k8s+Helm自动化部署实践（干货分享！）
通过前面两篇文章,我们已经有了一个"嗷嗷待哺"的K8s集群环境,也对相关的概念与组件有了一个基本了解(前期对概念有个印象即可,因为只有实践了才能对其有深入理解,所谓"纸上 ...
- 【k8s实战一】Jenkins 部署应用到 Kubernetes
[k8s实战一]Jenkins 部署应用到 Kubernetes 01 本文主旨 目标是演示整个Jenkins从源码构建镜像到部署镜像到Kubernetes集群过程. 为了简化流程与容易重现文中效果, ...
- kubernetes部署jenkins(Docker in Docker)及认证
引言 Jenkins是一款开源 CI&CD 软件,用于自动化各种任务,包括构建.测试和部署软件. 本文将Jenkins的master与slave置于Pod中,部署在namespace:jenk ...
随机推荐
- vue中将分号去掉，将双引号变为单引号的配置
在项目根目录下创建.prettierrc文件,文件内容如下: { "semi": false, "singleQuote": true } 实现vs code中 ...
- vue之v-for遍历下拉框select和单选框组radio-group
1.v-for遍历下拉框 <el-form-item label="审核状态:" prop="status"> <el-select v-mo ...
- 【资源下载】安卓VS鸿蒙第三方件切换宝典 V1.0
下载<安卓VS鸿蒙第三方件切换宝典> 由于字数较多,本文仅展示部分,查看完整版请点击上方下载 众所周知,安卓应用开发经过这么多年的发展相对成熟和稳定,鸿蒙OS作为后来者兼容一个成熟的开发体 ...
- Docker的架构
一.Docker引擎 docker引擎是一个c/s结构的应用,主要组件见下图: Server是一个常驻进程 REST API 实现了client和server间的交互协议 CLI 实现容器和镜像的管理 ...
- Android7.0无需FileProvide搞定URI拍照、应用安装问题
根据官方文档,从Android7.0版本开始 使用URI打开或安装文件需要单独在应用里配置了,问了度娘,有好多版本的结果,个人认为最靠谱的就是下边这个方法,只需在application的oncreat ...
- 如何用css写一个带斜切角、有边框又有内外阴影的按钮呢？
如果有一天,UI设计师丢过来一张UI稿,上面有这样一个带有斜切角.有边框还有内外阴影的按钮,你会怎么实现呢?第一反应切图?可是按钮内容.大小都是可变的,那得切多少图啊~Canvas?SVG?No,no ...
- springboot源码解析-管中窥豹系列之EnableXXX（十）
一.前言 Springboot源码解析是一件大工程,逐行逐句的去研究代码,会很枯燥,也不容易坚持下去. 我们不追求大而全,而是试着每次去研究一个小知识点,最终聚沙成塔,这就是我们的springboot ...
- 12. Vue搭建本地服务
一. 搭建本地服务器 本地服务可以提高开发效率. webpack不需要每次都打包, 就可以看到修改后的效果. 本地服务器基于node.js搭建, 内部使用二十express框架. 可以实现让浏览器自动 ...
- Java入门和环境配置ideaJ安装
Java入门及环境搭建 目录 Java入门及环境搭建 什么是Java Java Java的发展 Java的特性和优势 Java三大版本 JDK JRE JVM JAVA开发环境搭建 安装JDK 卸载J ...
- CentOS 8.3安装MySQL 8.0.21后无法登录管理数据库
安装mysql后登录不了,提示: ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES) ...