实战项目部署应用到kubernetes流程(jenkins+docker+k8s)
说明
通过jenkins构建java应用程序发布到k8s集群中
本文已一个大数据的java项目来演示构建部署过程
支持发布和回滚
支持一套模板应用不同项目
k8s基础准备
创建项目名称空间
[root@k8s-master ~]# kubectl create ns architect
namespace/architect created [root@k8s-master ~]# kubectl label namespace architect istio-injection=enabled
namespace/architect labeled
[root@k8s-master ~]# kubectl get namespace -L istio-injection
NAME STATUS AGE ISTIO-INJECTION
architect Active 87s enabled
default Active 39d enabled
istio-system Active 7d17h
kube-public Active 39d
kube-system Active 39d
[root@k8s-master ~]#
创建全局配置文件
＃创建大数据配置文件
[root@k8s-master ~]# ls bigdata-counter
core-site.xml dmp_operator1.keytab hbase-site.xml hdfs-site.xml krb5.conf
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl create configmap bigdata-config --from-file=/root/bigdata-counter/ -n architect
configmap/bigdata-config created
[root@k8s-master ~]#
#全局环境变量
[root@k8s-master ~]# cat yaml/global-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: global-config
namespace: architect
data:
CONFIG_SERVER_URL: http://config.ms.xx.com
CONFIG_SERVER_USER: myconfig
ILOG_BOOTSTRAP_SERVERS: bootstrap.servers=192.168.1.122:9092
CONFIG_SERVER_PWDS: mc123456 [root@k8s-master ~]#
[root@k8s-master ~]# kubectl apply -f yaml/global-config.yaml
configmap/global-config created
[root@k8s-master ~]# kubectl get configmap -n architect
NAME DATA AGE
bigdata-config 4 2m33s
global-config 4 3s
[root@k8s-master ~]#
创建habror认证信息
[root@k8s-master ~]# kubectl create secret docker-registry registry-secret --docker-server=dev-hub.xx.net --docker-username=admin --docker-password=Harbor12345 --docker-email=admin@dev-hub.xx.net -n architect
secret/registry-secret created
[root@k8s-master ~]# kubectl get secret -n architect
NAME TYPE DATA AGE
default-token-5wsxl kubernetes.io/service-account-token 3 2m24s
istio.default istio.io/key-and-cert 3 2m24s
registry-secret kubernetes.io/dockerconfigjson 1 10s
项目deployment文件
可以做成模本，通过变量替换形式．后面有讲到
[root@k8s-master ~]# cat bigdata-interface-deploy.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: bigdata-interface
version: default
name: bigdata-interface
namespace: architect
spec:
replicas: 1
selector:
matchLabels:
app: bigdata-interface
version: default
template:
metadata:
labels:
app: bigdata-interface
version: default
spec:
containers:
- env:
- name: izone
- name: MY_SERVICE_NAME
value: bigdata-interface
- name: ilogEnvs
value: izone,buildNumber
- name: ILOG_BOOTSTRAP_SERVERS
valueFrom:
configMapKeyRef:
key: ILOG_BOOTSTRAP_SERVERS
name: global-config
- name: CONFIG_SERVER_URL
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_URL
name: global-config
- name: CONFIG_SERVER_USER
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_USER
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: JAR_BOOT_ARG
value: --spring.cloud.config.fail-fast=true
- name: JDK_HEAP_OPTS
value: -Xms256m -Xmx256m
image: dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b63
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 2
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 300
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: bigdata-interface
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
volumeMounts:
- mountPath: /opt/jdk
name: jdk-path
- mountPath: /srv/applogs
name: applogs
imagePullSecrets:
- name: registry-secret
restartPolicy: Always
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /usr/local/jdk/default
type: ""
name: jdk-path
- hostPath:
path: /data/applogs
type: ""
name: applogs
[root@k8s-master ~]#
jenkins pipeline 模板文件
node {
//项目名称
String APPNAME='bigdata-interface'
//名称空间
String NAMESPACE='architect'
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(name:'GIT_BRANCH_TAG', defaultValue: 'master',description:'', trim: false)
])])
stage('git clone') {
checkout([$class: 'GitSCM', branches: [[name: params.GIT_BRANCH_TAG]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bc', url: 'git@git.jiatuiyun.net:bigdata-center/bigdata-interface.git']]])
}
stage('maven build') {
sh 'mvn clean package -U'
}
stage('docker build && push habror'){
sh "cd ${WORKSPACE}"
restapi = '`ls -d *-restapi||echo .`'
sh "ls -l ${restapi}/target/"
sh "echo ${restapi}"
sh "curl -o ${restapi}/target/Dockerfile http://192.168.1.55:8089/files/Dockerfile-java.v1"
sh "docker build -t ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${restapi}/target"
sh "docker tag ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
sh "docker login ${HarborUrl} -u ${HarborUser} -p ${HarborPasswd}"
sh "docker push ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
}
stage('deploy'){
echo "${params.GIT_BRANCH_TAG}"
echo "${BUILD_NUMBER}"
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
进一步改造pipeline
node {
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: 'git@git.jiatuiyun.net:bigdata-center/bigdata-interface.git', description: 'git仓库地址', name: 'XBUILD_REPO', trim: false),
string(defaultValue: 'master', description: '版本名称', name: 'XBUILD_BRANCHE', trim: false)
])])
stage('git clone') {
checkout([$class: 'GitSCM', branches: [[name: params.XBUILD_BRANCHE]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bc', url: params.XBUILD_REPO]]])
}
stage('maven build') {
sh 'mvn clean package -U'
}
stage('docker build && push habror'){
sh "cd ${WORKSPACE}"
restapi = '`ls -d *-restapi||echo .`'
sh "ls -l ${restapi}/target/"
sh "echo ${restapi}"
sh "curl -o ${restapi}/target/Dockerfile http://192.168.1.55:8089/files/Dockerfile-java.v1"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
sh "docker build -t ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${restapi}/target"
sh "docker tag ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
sh "docker login ${HarborUrl} -u ${HarborUser} -p ${HarborPasswd}"
sh "docker push ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
}
stage('deploy'){
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
echo "${params.GIT_BRANCH_TAG}"
echo "${BUILD_NUMBER}"
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
//updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
jenkins上创建自由风格项目
效果如下 支持参数化构建。但每项目单独JOB
查看镜像仓库已经推送正常
项目回滚
node {
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: '1', description: '镜像标签', name: 'XBUILD_NUMBER', trim: false)
])])
stage('deploy'){
echo "${BUILD_NUMBER}"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
BUILD_NUMBER = params.XBUILD_NUMBER
echo "----${params.XBUILD_NUMBER}---${params.XBUILD_DEPLOY} ----${params.XBUILD_TEAM} "
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
//updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
jenkins效果如下：输入namespace和deployment 名称和上次构建版本
查看k8s集群
可以灵活的在已知版本回滚操作
[root@k8s-master ~]# kubectl get deploy -n architect -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
bigdata-interface 0/1 1 0 3h12m bigdata-interface dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b1 app=bigdata-interface,version=default
[root@k8s-master ~]# kubectl get pods -n architect
NAME READY STATUS RESTARTS AGE
bigdata-interface-6bf7b5646d-sf9j5 1/2 Running 0 24s
[root@k8s-master ~]#
通过以上实践，可以将deployment做成模板
deployment模板文件
-bash-4.2$ cat k8s-deployment.tpl
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: {APPNAME}
version: default
name: {APPNAME}
namespace: {NAMESPACE}
spec:
replicas: 1
selector:
matchLabels:
app: {APPNAME}
version: default
template:
metadata:
labels:
app: {APPNAME}
version: default
spec:
containers:
- env:
- name: izone
- name: MY_SERVICE_NAME
value: {APPNAME}
- name: ilogEnvs
value: izone,buildNumber
- name: ILOG_BOOTSTRAP_SERVERS
valueFrom:
configMapKeyRef:
key: ILOG_BOOTSTRAP_SERVERS
name: global-config
- name: CONFIG_SERVER_URL
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_URL
name: global-config
- name: CONFIG_SERVER_USER
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_USER
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: JAR_BOOT_ARG
value: --spring.cloud.config.fail-fast=true
- name: JDK_HEAP_OPTS
value: -Xms256m -Xmx256m
image: {HarborUrl}/{NAMESPACE}/{APPNAME}:{NAMESPACE}_b{BUILD_NUMBER}
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 2
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 300
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: {APPNAME}
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
volumeMounts:
- mountPath: /opt/jdk
name: jdk-path
- mountPath: /srv/applogs
name: applogs
imagePullSecrets:
- name: registry-secret
restartPolicy: Always
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /usr/local/jdk/default
type: ""
name: jdk-path
- hostPath:
path: /data/applogs
type: ""
name: applogs
-bash-4.2$
通过模板动态生成项目的deploy文件．进行升级回滚．
比如下面的：
node {
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: '1', description: '镜像标签', name: 'XBUILD_NUMBER', trim: false)
])])
stage('deploy'){
echo "${BUILD_NUMBER}"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
BUILD_NUMBER = params.XBUILD_NUMBER
echo "----${params.XBUILD_NUMBER}---${params.XBUILD_DEPLOY} ----${params.XBUILD_TEAM} "
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
command="sed -e 's#{APPNAME}#${APPNAME}#g;s#{NAMESPACE}#${NAMESPACE}#g;s#{HarborUrl}#${HarborUrl}#g;s#{BUILD_NUMBER}#${BUILD_NUMBER}#g' /data/jenkins_slave/k8s-deployment.tpl > /data/jenkins_slave/${APPNAME}-deploy.yaml"
sh "${command}"
sh "scp -P 22 /data/jenkins_slave/${APPNAME}-deploy.yaml ${KubernetHost}:/root/ "
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
sh "ssh -p 22 ${KubernetHost} kubectl apply -f /root/${APPNAME}-deploy.yaml"
//test
}
}
pipeline参数化构建
示例：
pipeline {
agent any
parameters {
gitParameter branch: '', branchFilter: '.*', defaultValue: '', description: '', name: 'XBUILD_BRANCHE', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH_TAG'
string defaultValue: 'git@git.xx.net:company/xx-xx.git', description: '', name: 'XBUILD_REPO', trim: false
}
stages {
stage('Example') {
steps {
checkout([$class: 'GitSCM', branches: [[name: params.XBUILD_BRANCHE]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bb', url: params.XBUILD_REPO]]])
}
}
}
}
构建效果如下：
实战项目部署应用到kubernetes流程(jenkins+docker+k8s)的更多相关文章
- docker 实战-项目部署
前面学习了 docker 的命令和实际的用法,现在来实战一下.编排一个服务,也就是项目部署. 目前我们在一个闭源环境下工作,介绍一些工作流程和部署项目的方法. 该工作流程比较特殊 所谓闭源环境就是 没 ...
- jenkins+docker+k8s项目发布
目录 一.简介 二.新建docker-build工程 三.项目部署 四.访问测试 一.简介 1.该章节基于jenkins.Harbor.pipeline.k8s来做发布,如对这些不熟悉,请按以下进入学 ...
- .Net 6 WebApi 项目部署到 Linux 系统上的 Docker 容器
1.创建一个基础的WebApi项目 注意:因为发布时候,Dockerfile文件必须和解决方案.cspro文件放在同级,所以建议勾上这个,当时遇到这个问题,导致打包镜像时找不到.cspro文件,搞了好 ...
- 第四十四章 微服务CICD（6）- gitlab + jenkins + docker + k8s
总体流程: 在开发机开发代码后提交到gitlab 之后通过webhook插件触发jenkins进行构建,jenkins将代码打成docker镜像,push到docker-registry 之后将在k8 ...
- gitlab + jenkins + docker + k8s
总体流程: 在开发机开发代码后提交到gitlab 之后通过webhook插件触发jenkins进行构建,jenkins将代码打成docker镜像,push到docker-registry 之后将在k8 ...
- kubernetes下jenkins实战maven项目编译构建
关于kubernetes环境的jenkins集群 在kubernetes环境部署的jenkins集群,执行任务时会新建pod,任务完成后pod被销毁,架构如下所示: 在kubernetes搭建jenk ...
- Kubernetes笔记（三）：Gitlab+Jenkins Pipeline+Docker+k8s+Helm自动化部署实践（干货分享！）
通过前面两篇文章,我们已经有了一个"嗷嗷待哺"的K8s集群环境,也对相关的概念与组件有了一个基本了解(前期对概念有个印象即可,因为只有实践了才能对其有深入理解,所谓"纸上 ...
- 【k8s实战一】Jenkins 部署应用到 Kubernetes
[k8s实战一]Jenkins 部署应用到 Kubernetes 01 本文主旨 目标是演示整个Jenkins从源码构建镜像到部署镜像到Kubernetes集群过程. 为了简化流程与容易重现文中效果, ...
- kubernetes部署jenkins(Docker in Docker)及认证
引言 Jenkins是一款开源 CI&CD 软件,用于自动化各种任务,包括构建.测试和部署软件. 本文将Jenkins的master与slave置于Pod中,部署在namespace:jenk ...
随机推荐
- go 报 need type assertion
responese_total := m["responses"].([]interface{})[0].(map[string]interface{})["hits&q ...
- POJ-1860(最短路问题，Bellman-Ford算法判正圈)
Currency Exchange POJ-1860 这题其实是最短路问题的变形,但是这里不用求解最短路,而是求解路径中是否存在正圈.如果存在正圈则说明兑换后的货币可以一直增加,否则不能实现通过货币转 ...
- salesforce零基础学习（一百零一）如何了解你的代码得运行上下文
本篇参考:https://developer.salesforce.com/docs/atlas.en-us.228.0.apexcode.meta/apexcode/apex_enum_System ...
- Mock 框架 Moq 的使用
Mock 框架 Moq 的使用 Intro Moq 是 .NET 中一个很流行的 Mock 框架,使用 Mock 框架我们可以只针对我们关注的代码进行测试,对于依赖项使用 Mock 对象配置预期的依赖 ...
- CRLF注入漏洞 -配置错误
漏洞分析参考 https://i-beta.cnblogs.com/posts/edit 什么是CRLF? CRLF 指的是回车符(CR,ASCII 13,\r,%0d) 和换行符(LF,ASCII ...
- 几大BSD 区别
OpenBSD 侧重于安全,软件包较少,较陈旧,比如 KDE 才 3.5,为了安全舍弃了 sudo 和 linux 兼容层: FreeBSD 是开发者最多用户最多软件包最多的,有 ZFS 和 Linu ...
- mysql最权威的总结
1.数据库操作 create database person charset utf8; -- 创建数据库show DATABASES; -- 查看数据库drop database person; - ...
- Flutter 改善套娃地狱问题（仿喜马拉雅PC页面举例）
前言 这篇文章是我一直以来很想写的一篇文章,终于下定决心动笔了. 写Flutter的小伙伴可能都感受到了:掘金的一些热门的Flutter文章下,知乎的一些Flutter的话题下或者一些论坛里面,喷Fl ...
- 深入理解Java并发框架AQS系列（三）：独占锁（Exclusive Lock）
一.前言 优秀的源码就在那里 经过了前面两章的铺垫,终于要切入正题了,本章也是整个AQS的核心之一 从本章开始,我们要精读AQS源码,在欣赏它的同时也要学会质疑它.当然本文不会带着大家逐行过源码(会有 ...
- 【linux】驱动-3-字符设备驱动
目录 前言 3. 字符设备驱动 3.1 Linux设备分类 3.2 设备相关概念 3.2.1 设备号 3.2.2 设备节点 3.2.3 APP open 文件理解 ** 3.3 数据结构 3.3.1 ...